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FOREWORD 



The purpose of this study was to selectively review the 
literature on the evaluation of advertising effectiveness and to 
examine the relationship between advertising and enlistments in the 
Navy in order to achieve a better understanding of the Navy’s 
advertising programs and the utilization of its resources. The work 
was supported by the Navy Personnel Research and Development Center, 
San Diego, California, under its acquisition and initial service 
programs headed by Dr. Martin F. Wiskoff. The project monitor was 
Dr. Robert A. Dakota who provided valuable guidance and suggestions 
for the program and was most patient and understanding. Similar 
help and cooperation was also received from Dr. Robert V. Guthrie. 

The empirical portions of this study could not have been 
completed without the full cooperation and support of the Navy 
Recruiting Command. I am particularly indebted to LCDR H. A. Levien, 
LCDR P. K. Vanwinkle, and Mr. Wade Badger of the Research and 
Analysis Division, Plans and Policy Department, and to LCDR W. T. 
Shiffer, LCDR J. Fyfe , Ms. D. Edwards, Mrs. S. Motley of the Manage- 
ment Systems Division, Recruiting Advertising Department. The 
report on national media expenditures prepared by Ted Bates, Inc., 
and sponsored by the Navy Recruiting Command, was an absolutely 
essential component of this study. Nancy Clott and Jackie DeCosta 
were the account and research heads, respectively, for this effort. 

The opportunity to interact with the Joint Market Analysis 
Research Committee sponsored by Dr. Albert J. Martin, Director of 
Acquisitions in the Office of the ASD (MRA&L) and headed by its 
chairman, Ronald G. Liveris of the US Army Recruiting Command, 
provided much information and a broad outlook on the marketing 
problems facing the Armed Forces. The Defense Manpower Data 
Center (DMDC) was also most helpful in providing essential enlistment 
and environmental data and the self-report data from the Market 
Facts surveys. Dr. John R. Goral and MAJ (USAF) Rex L. Klaurens 
require special mention. 

An interim product in this project was the report of January 
1978 prepared by Dr. Frederick C. Nold and Dr. Michael K. Block of 
Rhodes Associates entitled "Evaluation of Navy Advertising Effec- 
tiveness." The study was conducted prior to the availability of 
the Ted Bates, national media report and attempted to augment the 
then sparse, recruiting command data with the self-report data 
provided by the Market' Facts ’ "Youth Tracking Surveys" carried out 
for the Department of Defense. The report provided valuable insight 
into the dynamics of the recruiting advertising environment, problems 
in its evaluation, and especially, the need for detailed, appropriate 
data. 
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Finally, I must express my deep gratitude and indebtedness to 
LT Thomas C. Williams, Jr. for organizing and analyzing the data 
used in Part II of this study and providing valuable suggestions and 
observations in conjunction with his excellent master's thesis in the 
Operations Research curriculum of the Naval Postgraduate School. 

Mrs. Pat Meadows, programmer for the Departments of Operations Research 
and Administrative Sciences, was, as always, most helpful. Ms. Sunny 
Matteson did her usual conscientious job in typing the manuscript. 
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SUMMARY 



Since the advent of the all-volunteer force (AVF) , Navy adver- 
tising expenditures for recruiting have increased approximately 10 
times over ' previous years in constant dollars and averaged 25.1 
million in actual dollars during the first three years of the AVT. 

As a result of Congressional concern over the size of these expendi- 
tures, advertising budget levels have been reduced considerably since 
then, but there is the continuing question of how much is actually 
needed. Any answer to the question is difficult to justify because 
the relationship between advertising and enlistments is not known. 

The problem of determining advertising budgets has existed for 
a much longer time in the civilian sector without definitive answers. 

On the basis of much experience, many firms may have some idea as to 
what is adequate and where the upper and lower bounds may be, but the 
actual budget decision is generally some constant proportion of a pro- 
duction or sales decision. Since organizations usually come close to 
their projections, the advertising budget decision appears to be 
validated. 

Experimental or data-based approaches to determine the relation- 
ship between advertising budgets and sales, or some other bottom-line 
measure, have only met with limited success. Where budgets have been 
deliberately manipulated in control or levels experiments, the presence 
of a great many other concurrent effects on the output measure has made 
the results equivocal. The alternative is a much larger number of 
test areas or a long sequence of repeated experiments. In both cases, 
the need is to increase the number of data points, because the method 
tends to be very data limited. 

Where data-based approaches have been used to take advantage of 
naturally occurring experiments, the primary problem has been in determin- 
ing cause and effect relationships. This problem is not insurmountable, 
but the data requirements and analytical procedures are quite formidable. 

A not easily surmountable problem occurs when advertising is found to 
have no reliable effect on the output measure. In this case, the method 
cannot determine what the effects might be if advertising expenditures 
considerably different from those that were found to occur naturally 
were to be implemented. Nevertheless, the data-based approach is 
usually not data limited, and it has the advantage of not interfering 
with or causing a reaction in the process being evaluated. 

When those actually engaged in the advertising process — agencies 
and advertising research firms — perform research on the effectiveness 
of advertising, they tend to be satisfied with intermediate output 
measures, such as awareness of and attitude toward the product or 
service being advertised. Rather than budget levels, their concern is 
with the message, the segmentation of the population into target 
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audiences, and the reach and frequency with which the audiences are 
contacted. The bridge between awareness and attitudes on one hand, 
and sales, say, on the other is not often crossed. Moreover, the 
rationale for the population segmentation and the appropriateness of 
the message are usually not validated in terms of their effect on the 
customer's bottom-line measures. The Navy's advertising program in 
the national media conducted through its agencies has many of these 
characteristics . 

In addition to agency advertising through the national media, 
the Navy internally operates a local advertising management system 
(LAMS) and creates and distributes recruiting aids (RAD items) . There 
is also a public service advertising program in which the media are 
solicited to provide free space or time for Navy recruiting advertise- 
ments. Control experiments have not been attempted, although they 
have been recommended and planned. Data-based approaches were largely 
inappropriate because of data limitations with respect to national 
media advertising. This problem was corrected when the Navy Recruiting 
Command contracted with Ted Bates to reallocate or assign all national 
advertising expenditures in calendar years 1976 and 1977 to individual 
counties by month, media, and advertising programs — such as general 
enlisted, minority, and officer. The completion of this project made 
the empirical portion of the study reported herein possible. 

The purpose of the empirical study was to gain an understanding 
of the relationship between advertising and enlistments in the Navy in 
order to make recommendations for the evaluation of its advertising 
effectiveness and the making of budget levels decisions. The Ted Bates 
national media data for general enlisted programs were supplemented by 
Navy and DOD data allocated to the county level in proportion to the 
QMA. The LAMS and RAD expenditures were only available by quarter, so 
they were allocated to the three months within a quarter evenly. The 
national, monthly, unemplo 3 mient rate for youths aged 16-19 years was 
also obtained and assigned to all counties. In all, the most essential 
variables for examining the relationship between Navy advertising and 
enlistments were available at a very low level of aggregation — the 
county by month. These data included: 

Number of first-term male (QUEBEC or Q) enlistees 
Number of first-term male enlistees who were high-school 
graduates with AFQT scores equal to or greater than 50 
Qualified military availables (QMA) 

Enlistment (QUEBEC) goals 

Number of canvassers 

Total media expenditures 

Total LAMS expenditures 

Total RAD expenditures 

Total of all advertising expenditures 

Number of high-school graduates 

Unemployment rate 

Year (1976, 1977) 
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The advertising expenditures were also lagged by 1 and 2 months so 
that the carryover effect of current advertising could be evaluated 
1 and 2 months later. The enlistees who were high-school graduates 
and had AFQT scores of 50 or better were considered to be "quality" 
enlistees. All enlistees were assigned to the month in which they 
actually enlisted — the "contract signed" date — and not the often, 
erroneously used date when they shipped to the RTC. 

Some outlier cases originating from very small counties were 
dropped, as were county-months that had no enlistments. All cases for 
December 1976 were dropped as outliers because enlistments were more 
than double those of previous months due to the pending, year-end 
termination of Veterans' benefits. These figures were: 

Average for First 11 Months Q Enlistments in December 
7,737 16,457 

An examination of the remaining raw data showed very high 
commonalities among all of the key input and output recruiting variables, 
and all were closely related to the QMA. The percent of variation in 
these variables predictable from a knowledge of the QMA, alone, was: 

Variable Percent Predictable 

Average of advertising categories 51.8 

Total Q enlistments 84.6 

Q enlistment goals 84.6 

Number of canvassers 96.0 

The amount of variation in Q enlistments that was predictable from a 
knowledge of advertising variables averaged 42.2 percent. These results 
indicated that, as in civilian industry, a major portion of the input- 
output relationship is predetermined by management planning which, in 
this case, relies heavily on the QMA as a planning base. Any analysis 
or research that uses only the raw data would be capitalizing on, or 
merely recapturing, management strategy. 

To escape this situation the key variables were normalized or 
indexed by the planning base, the QMA. They then became rate variables 
that could vary independently of the QMA. When this was done to the 
study data, the previously observed relationships were highly attenuated 
and the relationship between advertising and Q enlistments disappeared 
completely. This was found to be due to a highly accelerated growth 
of the advertising rate compared with the enlistment rate. As adver- 
tising rates grew larger, they were not accompanied by similar increases 
in the enlistment rate. 

A fourth root transformation of the advertising variables before 
they were divided by the QMA established a linear relationship between 
the advertising rate and the enlistment rate. The rate variables were 
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then entered into an ordinary least squares, linear regression model 
to determine the relationships between the input variables considered 
simultaneously and the two output (enlistment) variables considered 
independently. It was found that 55.6 percent of the variation in Q 
enlistment rates and 73.7 percent of the variation in quality enlisted 
rates could be predicted using 11 and 10 input variables, respectively. 
These variables, in order of the magnitude of their effects on the 
enlistment rates, were: 

1. The three transformed advertising variables in their current 
and lagged 2-month forms 

2. The proportion of high school graduates in a county 

3. The year in which the county-month appeared 

4. Canvassers per QMA 

5. Unemplo37raent rate, which had a small, but negative effect 

The advertising variables had the greatest effect, by far, in 
predicting the enlistment rates. These results indicate that a close 
relationship between the advertising rate and the enlistment rate, 
especially in the case of the important quality enlistees, can be 
achieved by a simple change in the distribution of advertising and 
that this relationship is much closer to the enlistment rates than 
the distribution of canvassers or goals. That is, an increment in the 
advertising rate had a much greater increment in the enlistment rate 
than a similar increment in canvassers per QMA or goals per QMA. 

The importance of the high-school graduates per QMA in predicting 
the enlistment rate confirmed previous research by the author which 
showed the sensitivity of enlistments to the educational quality of an 
area. 



The contribution of the year variable is important because it 
raises questions about the allocation of advertising resources and 
effort. Except for one month of radio, 1976 had no paid advertising in 
the electronic media. The following year, however, had a full program 
of paid radio advertising, and paid television advertising began in 
April 1977. The electronic media accounted for a majority of the 
media costs. A comparison of the years shows: 

Item 1976 1977 

Total Q enlistments 101,568* 88,556 

Quality enlistments 51,309* 37,495 

Average per month canvassers 3,212 3,360 

Media expenditures (calendar year) $3,916,371 $5,803,669 

*Includes December 1976. 

Goals per QMA was a variable that was conspicuous by its absence 
in affecting the enlistment rate. This was attributed to the influence 
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of the DEP status and goal accomplishment in the near past as 
contributing factors, along with goals, in driving the recruiting 
effort. A better measure than just goals was required to account 
for some of the unexplained variation in enlistment rates. 
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RECOMMENDATIONS 



The following recommendations are made on the basis of the 
selective literature review and the background research reported in 
Part I of this study and the findings of the empirical analysis 
reported in Part II. 

1. Whenever the temporal coincidence of recruiting operations — 
such as advertising and current production — must be established for 
research or management purposes, the date of actual enlistment — i.e., 
a ^contract signed” date — should be used for the production measure; 
research products or managerial decisions that use production measures, 
such as shipments to the RTCs, must be considered highly suspect when 
there is a requirement for temporal coincidence. 

2. Measures of recruiting inputs and outputs should be made 
into rate measures reflecting qualitative aspects of the operations 
for research purposes; research products that do not do this should 
be highly suspect, since the relationship among raw measures of key 
recruiting variables are primarily predetermined by management decision. 

3. Control experiments to evaluate the effects of budget levels 
should take into consideration a large number of contemporaneous oper- 
ational and situational variables to ensure unambiguous and general- 
izable results; this implicates the need for a large number of test 
areas and tests of sufficient duration. 

4. Media mixes and the allocation of advertising resources 
among national media, local advertising, and recruiting aids should 
be examined to determine the optimal distribution of effort among 
these alternatives; specifically, the utility of paid radio and 
television advertising relative to other forms of advertising and 
promotions requires immediate and close examination. 

5. Research should be undertaken to determine the effects on 
enlistments that varying the advertising rate may have, given the 
current potential of an area; that is, should larger rates of advertis- 
ing be devoted to already highly producing areas or should they be 
addressed to improving poor areas? 

6. Assumptions, guidelines, and policies for the conduct of 
advertising operations should be critically examined for the evidence 
on which they are based and the actual results they are producing; 
more specifically, the manner of calculation and utilization of the 
Recruitment Development Index (RDI) should be examined to determine 
their consequences . 

7. The effects of goals, the DEP status, and, possibly, goal 
attainment in the immediate past should be identified and combined 
into a single measure that describes a force that drives enlistments 
in an NRD at a particular point in time. 
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8, An attempt should be made to develop a comprehensive, 
testable, and useful model that takes advertising as a communication 
process involving meassage, reach, frequency, and the target audience 
and relates this process to hard measures of recruit production. 

9. Advertising expenditures should be documented in the monthly 
enlistment summary of the Navy Recruiting Command so that the relation- 
ship between expenditures and enlistments can be monitored. 
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PART I . BACKGROUND 



ADVERTISING BUDGET DECISIONS, EVALUATING ADVERTISING 



EFFECTIVENESS, AND NAVY ADVERTISING PROGRAMS 



INTRODUCTION 



Orientation of the Study 

The need to determine the effectiveness of advertising is a 
matter of major concern in any marketing effort (Lipstein & McGuire, 
1978). It is no different in the case of military advertising for 
recruiting purposes (Comptroller General, 1976). The All-Volunteer 
Force (AVF) came into existence in July 1973, and it has presented a 
major challenge as a marketing effort. Enlistment contracts must be 
marketed and sold in large numbers to staff the military forces ade- 
quately in the total number of personnel and in the requisite quality. 
The competition to attract quality youth entering the labor market 
involves not only the entire civil sector, but there is also keen 
competition among the services as well. Thus, the services compete 
with the civil sector in the context of the society-at-large, and 
there is a share-of-the-market competition within that subset of 
American youth that is willing to consider the military services for 
an entry-level job. Complicating the market-share picture are vari- 
ations in the requirements, attractiveness, and costs (negative aspects) 
of service in any one of the Armed Forces. 

The purpose of this study was to examine the Navy's advertising 
in this competitive market with no delusions about finding a definitive 
answer to the question of the effectiveness of the Navy's advertising 
effort. Rather, the objectives of the study were to become familiar 
with the structure and dynamics of Navy advertising, examine available 
data for possible trends between advertising and response measures in 
the target audience, and to make recommendations for a program of 
increasingly definitive studies to determine the effectiveness of 
Navy advertising and to suggest means for increasing its effectiveness. 

Magnitude of Navy Advertising 

A record of advertising expenditures by the Navy over the last 
fifteen years is presented in Table 1. There it can be seen that 
spending on advertising was just a little more than a million dollars 
in FY 1964. This was less than .01 percent of all Navy expenditures 
during the fiscal year. Expenditures, in constant dollars, remained 
very stable in subsequent years, even with the large increase in per- 
sonnel requirements as a result of the war in Vietnam. High levels of 
advertising were unnecessary because individuals faced with conscription 
and probably service in one of the combat arms of the Army — "voluntarily 
enlisted for longer terms of service in order to have a choice of 
service and, possibly, a choice as to area of training and specializa- 
tion. In FY 1972, however, with the peak involvement in Vietnam (FY 
1969) past and the prospects of the AVF a reality. Navy advertising 
expenditures increased almost fourfold (in real dollars) over those in 
FY 1971. This was followed by a threefold increase in FY 1973 over 
the FY 1972 expenditures. It was FY 1973, of course, which saw a 
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Table 1 



Total Navy Advertising Budget 
(in thousdands of dollars) 



Current Dollars 



Constant Dollars 





1,004 


1,088 




919 


981 




1,029 


1,072 




1,324 


1,342 




1,298 


1,271 




1,465 


1,369 




1,667 


1,474 




1,798 


1,513 




7,051 


5,719 




23,017 


17,815 




26,753 


19,055 




25,549 


16,537 




16,839 


10,150 




16,389 


9,228 




18,030 


9,836 


Navy 


Recruiting Command. 




Year 


1967 = 100 





Current dollars and constant dollar figures are 
not identical in 67 because expenditure figures 
are FT 67 and the deflator is 100 in CY 67. 

Change in start of fiscal year from July to October. 
Advertising expenditures during the transitional quarter 
(TQ) were $3,318,000 in current dollars and $1,964,000 
in constant dollars. 
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major shift in the defense posture with the withdrawal from Vietnam 
and the conclusion of the draft. Recent years have brought about a 
considerable drop from the peak expenditures in FY 1974, the first 
complete year of the AVF, and expenditures seem to have plateaued at 
about six times the level of the last year completely unaffected by 
the AVF (FY 1971). 

Even the peak expenditures for advertising in FY 1974 repre- 
sented no more than .5 percent of direct personnel costs and .1 per- 
cent of total Navy appropriations for that year. While the absolute 
amounts may seem to be considerable, the relative costs for advertising 
cannot be said to be a significant portion of the Navy’s budget. 

Another perspective for assessing the magnitude of the Navy’s 
advertising effort would be to compare expenditures with recruiting 
goals, the recruits and reenlistments attained, and the support provided 
the canvasser in the field. Table 2, therefore, shows the total goals 
for all enlisted programs, both USN and USNR, since the AVF was in- 
stituted and the attainment of those goals. Also shown is the size 
of the canvasser force that produced the enlisted acquisitions. The 
expenditures in Table 1 include those for officer programs so that the 
amount earmarked for enlisted programs is somewhat smaller. In gross, 
"ballpark" figures, somewhere between lOO to 200 dollars in advertising 
has been spent for one recruit, or, from the standpoint of the re- 
cruiter, approximately 5,000 dollars are spent per year in support of 
one canvasser in the field. While these figures are not unduly great, 
on an individual basis, they do add up to a multimillion dollar program 
of Navy advertising and are greater than unit advertising expenditures 
for durable items, such as automobiles. Thus, there is a basis for 
questioning the effectiveness of the advertising program, either from 
the total amount expended or the results that are produced. 
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Table 2 



Navy Goals, Goal Attainment, and Canvasser Resources 
in the AVF Period* 



Fiscal Year 


Goal 


Obtained 


Canvassers 


1974 


100,050 


101,346 


3,576 


1975 


112,130 


113,027 


3,760 


1976 


103,325 


103,587 


3,158 


1976(TQ) 


35,779 


33,887 


3,266 


1977 


116,314 


111,557 


3,477 



*Includes all USN and USNR, first-term and reenlistees, 
men and women. Source: Fiscal Year Navy Recruiting 

Command program summaries . 
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PROBLEMS IN EVALUATING ADVERTISING EFFECTIVENESS 



General Considerations 



Whereas the need to assess the effectiveness of advertising is 
universally acknowledged, there is much less agreement as to how it 
should be done* At the outset, the distinction between advertising 
as a communication process and advertising as a part of a marketing 
program must be recognized. A great deal of what has been called 
advertising research has been, and continues to be, undertaken in the 
former area. There is a plethora of advertising research firms that 
would undertake projects in this context. The basic ingredients for 
such research are a very detailed segmentaion of the population to 
identify the target audiences and then an evaluation of the reach and 
frequency of the advertising campaign in contacting that audience. 

Since the validity of the original segmentation process is frequently 
not empirically established but based on intuition, conjecture, and 
faith (Jacoby, 1976), this type of research is basically assessing the 
efficiency of the campaign and not its effectiveness. 

When the research attempts to determine whether the intended 
message was communicated and, additionally, whether a desired change 
in attitude occurred, then it can be considered to be evaluating the 
effectiveness of advertising. Those who consider advertising to be a 
communicative process may stop here. But there are the many who insist 
that adv-ertising, to be considered effective, must contribute to the 
objectives of the marketing program of which it is a part. One indi- 
cation of this effectiveness might be that it initiated action — e.g., 
contacting the recruiting station or visiting the product dealer. Those 
who would stop here insist that so many other factors enter the picture 
at this point that it would not do justice to advertising to insist 
that it must also account for the consumatory response such as the 
purchase of a product or a signature on an enlistment contract. This 
leaves the large group of persons who insist that advertising, to be 
considered effective, must contribute to the objectives of the organi- 
zation — i.e., that it must show an effect on the ’^bottom line.** The 
significance of this position, beyond that of basic management philos- 
ophy, arises when apparently effective communicative processes — those 
that inform and change attitudes — show no effect on the so-called 
bottom line. In fact, the situation may be just the opposite when, 
for example, initiating action or making a consumatory response may 
change attitudes and the possession of product information. Accordingly, 
it would seem advisable to insist that advertising effectiveness must 
be revealed through its contribution to marketing program goals and 
that it does this in the direction specified by advertising as a 
communicative process. 

The Simultaneity Problem 

The simultaneity problem arises when an outcome measure, such as 
sales, and the advertising effort — say, in dollars — covary because 
they are mediated by a common factor. That common factor is planning. 
Based on an estimate of demand and the production and marketing 
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capabilities of the organization, a sales target is established and 
resources are budgeted and allocated accordingly (Little, 1975a). 

When planned sales are up, so is the advertising budget; when retrench* 
ment in marketing plans occurs, the advertising level goes down. 

Thus, advertising is correlated with sales, and the unwary researcher 
may be trapped into positing a causal relationship from advertising to 
sales. Schmalensee (1972) analyzes this problem at length, cites 
studies that have ignored the existing simultaneity and shows the 
resulting ambiguity of the study conclusions for not having adequately 
considered the bidrectional relationship between advertising and sales. 

A similar condition has been alluded to, above, with respect to 
attitude change and consumer response. The direction of the effect of 
one on the other is very frequently in question. Social psychologists 
have shown that significant attitude change can occur after a response 
is made (Kiesler, Collins, & Killer, 1969). The cognitive dissonance 
concepts of Festinger (1957), for example, use a change in attitude to 
reduce the dissonance or conflict that may arise after one’s commit* 
ment to a particular, behavioral course of action. The result is that 
the favorable attitudes toward a product that may occur after the pur- 
chase of the item could be considerably greater than any favorable 
attitude existing prior to its purchase. 

The same situation may exist with awareness measures. That is, 
the unprompted recall of an ad may be more probable for persons who 
have purchased the product, but this heightened sensitivity to the 
advertising could well have occurred after the purchase, rather than 
before, as often assumed. For example, the recall of a new Virginia 
Slims cigarette ad may be greater among those who already smoke Vir- 
ginia Slims than among those who might have been induced by the ad to 
try a pack of the cigarettes. 

One way to clarify the direction of effect is through the use of 
simultaneous equations (Bass, 1969; Schmalensee, 1974). The principle 
is to have equations for each possible direction of effect involving 
the variables in question. Say that the variables of interest are 
advertising expenditures and sales. Then, each must appear as a de- 
pendent and an independent variable in separate equations to permit 
comparison of their relative effects vis-a-vis each other. The 
analysis, to be meaningful, has to be dynamic. Data from different 
points in time are required to unscramble the directionality question. 
Dhalla (1978) refers to this strategy as the 2-stage, least squares 
method. Thus, data requirements are considerably greater when simul- 
taneous equations are used. 

Aaker and Day (1974) have used the simultaneous equation 
method to investigate the relationships among advertising, consumer 
awareness, attitudes, and behavior (the purchase of instant coffee 
brands). They called their set of equations recursive, and brought 
out the fact that purchase of a brand of instant coffee predicted the 
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favorability of attitude toward that brand as well or better than 
the prediction that a favorable attitude toward a brand would result 
in its purchase. Their results also indicated that awareness of ad- 
vertising — its informative aspects — can be directly related to sales 
without being mediated by a corresponding change in attitude as hy- 
pothesized by most communication models of the advertising process 
(Engel, Kollat, & Blackwell, 1973; Kassarjian & Robertson, 1973). 

Sociologists and psychologists have used similar designs in 
what has been called cross-panel and path analysis methods (Blalock, 
1964; Yee & Gage, 1968; Simonton, 1977; Young, 1977). Thus, there is 
considerable guidance available to be aware of the problem of simul- 
taneously determined effects in advertising and to make appropriate 
adjustments in the research design. To recognize the situation when 
it exists may be more difficult, and it may not be possible to cor- 
rect for its presence. That is, limitations in the data may not per- 
mit the application of appropriate evaluative techniques. Finally, 
there is another way to circumvent the problem of inferring causation, 
and that is the controlled experiment (which will be discussed later) . 

In addition to the problem of endogenous and exogenous variables 
affecting desired outcomes simultaneously with advertising, there is a 
similar problem in advertising’s effects on itself. One manifestation 
of this problem is a residual effect of advertising that carries over 
into the current period to affect current outcomes along with current 
advertising. From the managerial standpoint, advertising in this per- 
spective becomes an investment, rather than merely a current expense. 

The typical method for assessing the carryover effect has been through 
the use of distributed lags in a regression equation in which advertis- 
ing in prior periods enters the equation as a potential determinant of 
current outcomes (Palda, 1964) . A problem that is more difficult to 
unravel and that tends to be situation specific with no general solu- 
tion is the overlap of advertising in the target audience when several 
media are used simultaneously. These temporal and geographic problems 
of the effects of advertising on itself are characteristic of the Navy’s 
advertising effort. 

Finally, there are other characteristics of the advertising 
process, itself, that have an effect on the effectiveness of adver- 
tising. These include such factors as the copy, media selection, the 
distribution of advertising among the media, and the temporal aspects 
of a campaign, such as the frequency and spacing of ads and its overall 
length. 

Management Decision-Making as an Evaluation Objective 



Discussed above was the purpose of advertising conceived as a 
communication process that begins with informing the potential customer 
about a product and eventually proceeds through attitude change and 
action initiation to product purchase. Krugman (1975) seeks to ex- 
plain the cognitive processes of the potential consumer during this 
sequence as a consequence of repeated exposures to an ad and hypo- 
thesizes the relationships between this covert behavior and advertising 
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that will make advertising effective. This approach is concerned 
with advertising, itself, as the object of study. The purpose of 
advertising may, however, be considerably different when examined 
from the standpoint of management. 

While maintaining or increasing sales may be the commonly 
stated objective of advertising, there are other management considera- 
tions of equal or greater importance. Share of the market, for ex- 
ample, ig probably of greater concern than an absolute level of sales. 
Carried to the extreme, this objective may have the effect of stifling 
or destroying competition. Whether and to what degree this is possible 
in an industry has been the subject of concern, critique, and conjec- 
ture (Comanor & Wilson, 1974; Ferguson, 1974). The possibility that 
excessive advertising by the services might seriously disrupt the 
marketplace for youth just entering the job market may have been be- 
hind the restriction placed by Congress in the early years of the AVF 
that prevented the services from purchasing advertising time on the 
electronic media. Creating good will and a favorable image may be 
other objectives of management. Much institutional advertising is of 
this nature. But whatever the stated objectives may be, the management 
problem common to them is the need for a budget decision. As a result, 
much research in advertising has been devoted to the process of making 
a budget decision. Then, research strategy may become considerably 
changed when the evaluation of advertising's effectiveness is subor- 
dinate to — actually, an input to — a budget decision question. The 
question, in its broadest sense, is all-pervasive and asks: How 

much is enough? 

One approach toward answering such questions is by building 
models that simulate the advertising process (Aaker & Myers, 1975; 
Dhalla, 1977; Little, 1966, 1975a, 1975b; Rao, 1970; Shane 1977). 

Most models however, are normative. They are analytical, provide in- 
sights into the advertising process, and elicit guidelines for plan- 
ning advertising programs and budgets. Such theoretical concepts as 
the shape of the response curve to advertising, the existence of a 
threshold level below which advertising produces no effects, the 
possibility of a downward bend in the curve with excessive advertising, 
and the shape and slope of the sales degradation curve with cessation 
of advertising lead to a host of decision possibilities that may in- 
volve criteria such as minimizing expenses, distributing expenses in 
an optimal manner over various media, maximizing exposure, or distri- 
buting expenditures optimally over time — such as by pulsing. The 
problems with these models for the specific case are the uncertain 
validity of the model for the particular decision and the lack of real- 
world data for practical application of the model. The former case is 
the well-known one of a solution looking for a problem. Correlating 
real world end states or outcomes with model predictions, as Shane 
(1977) does with election results predicted on the basis of relative 
advertising expenditures, does not validate the model, although the 
results may be reassuring. What is essentially needed is a demon- 
strated correspondence between the path-time histories of the input- 
output transformations in both cases to ensure the fit between the 
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model and the real world (Naylor & Finger, 1967; Van Horn, 1971). 

The problem in doing this resolves itself into the second of the prob- 
lems mentioned above, the lack of real-world data. 

The lack or even complete absence of real-world data to apply 
or calibrate a model results in a need to conduct experiments to 
obtain that data. These experiments have been referred to as levels 
experiments or control experiments, reflecting the process of actually 
manipulating budget levels and comparing them against control condi- 
tions to generate the needed data points. Little’s (1966, 1975a, 

1975b) models are self-adjusting by sampling regions around the pre- 
dicted sales-response curves with budget-level trials. Dhalla (1977) 
also requires either, or both, control experiments and "minitests" 
to validate and adjust his model’s outputs for setting advertising 
budget levels. There is a compelling need that is felt by management 
in the Navy’s case to generate similar data because there is no good 
answer to the annual question raised by Congressional committees as to 
how much is enough. 

In theory the controlled experiment is appropriate and even 
necessary. In practice, very close examination of the situation and 
considerable thought should be devoted to the decision of embarking 
on a course of control experiments. The reason is that the payoffs 
may not be worth the costs. The problem stems from the incompatible 
needs to keep the experiments from interfering as little as possible 
with ongoing operations and to execute a substantial number of repli- 
cations to provide confidence in the results. Specific problems may 
include the following. Management will permit experimentation only in 
peripheral areas to prevent harm from occurring to the key producing 
areas and to maintain revenues, resulting in a lack of representa- 
tiveness in the sample. There may be an insufficient duration of 
the tests — especially at low levels of support — to dissipate the carry- 
over effect from a previous level of support because of management’s 
fears of permanently damaging a market. There is very likely to be 
an insufficient number of territories to evaluate or control for some 
of the determinants of sales or productivity other than expenditures 
for advertising. In fact, there may not be a sufficient number of 
territories to evaluate the primary effects of budget levels if the 
variation in productivity within each budget level tends to be large 
with respect to the number and spacing of the budget levels designed 
into the experiment. The effects on the area sales force of manipu- 
lating the advertising budget seem never to be controlled or evalu- 
ated. Thus, it is not known whether greater productivity, if it 
occurs, was due to an increase in advertising level or an increase in 
worker-management level of effort because greater productivity was 
obviously expected of them, or some of both. Similarly, a lowered 
budget level could motivate the field force to work harder because it 
obviously has to in order to maintain productivity or just to show 
what it can do without support from above; it might also result in 
lowered effort. 
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Perhaps the best-known program of controlled experimentation is 
that conducted by Anheuser-Busch (A-B) over the period 1962 to 1968 
(Aaker & Myers, 1974, pp. 60-61). A portion of these experiments is 
reported in detail by Ackoff and Emshoff (1975) and Rao (1970); 
critical reviews of these reports are provided by Allaire (1975) and 
Ted Bates (Note 1). In these experiments, advertising budgets in 
some experimental areas were decreased to essentially zero and in 
others, increased by 200 percent or more to completely dominate the 
advertising of beer in the area. The results of the manipulations 
produced relatively small deviations from projected sales. Sales, in 
many instances, were found to rise or fall with reductions or in- 
creases in advertising. Areas were apparently able to meet projected 
sales levels with essentially no advertising support. Many explana- 
tions were offered for the findings but they were not based on a 
scientific evaluation of the experiments. Controlled experimentation 
during this period was conducted in series of ”mini experiments” and 
the statistical evaluation of the results almost generally showed no 
significant effects of advertising budgets. Allaire (1975) attributed 
this to the presence of too many other factors influencing sales so 
that the unexplained variance (error) overwhelms any effects that 
might have resulted with the manipulation of budget levels. Some of 
these other influences could have been the variability in the price 
of beer, the effects of competitive advertising, a drastic reduction 
in the number of competing breweries (from 170 to 70 during the period 
1960 to 1971), and a change in the distribution of advertising dollars 
over media — e.g., from 20 percent to zero for outdoor advertising, from 
5 percent to 50 for network television in the period 1961 to 1974 
(Allaire, 1975; Ted Bates, Note 1). More recently, the beer industry 
experienced a dramatic shift in advertising and product image due to 
Miller's overwhelming success in emphasizing its “light beer.” Under 
such conditions, even increases in advertising levels using the same 
old themes would have shown a decrease in sales or market share for 
competitors. 

One general, although important, decision by A-B management based 
on the experimental program was that it could decrease its advertising 
expenditures, which it did to a very considerable degree (Aaker & 

Myers, 1975; Ackoff & Emshoff, 1975b). The assumption was that A-B was 
advertising at a supersaturation point, and this could have accounted, 
in part, for the inconclusive results of the experiments. The conclu- 
sions to be drawn from the A-B program with respect to the utility of 
controlled, budget-level experiments is to use them for testing the 
extremes or limits of budgeting to answer a question such as ”How 
much is too much?” In fact, budget-level tests would be necessary 
in most cases to answer this type of a question because such extreme 
instances are not likely to be found in existing operational programs. 
But if the question to be answered by the research requires greater 
sensitivity as to advertising’s effects on an ongoing program, avail- 
able data generated by the operational program should be used. This 
usually results in a regression approach. Where desired data are not 
available, data generation requirements should be instituted and insti- 
tutionalized. The reason for this conclusion is that there are so 
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many factors other than advertising that affect outcomes that it is 
neither practicable nor possible to design controlled experiments that 
can include these effects in the design. On the other hand, there is 
considerable freedom in including variables in a regression approach as 
long as the data base is sufficiently large. A combination of the ap- 
proaches might be possible in which the advertising variable has been 
deliberately manipulated in a regression or similar evaluative model. 

Level of Investigation 



The problems discussed above relate to the evaluation of adver- 
tising effectiveness in the aggregate using macro approaches. The 
effects of advertising are studied over large classes of people, such 
as housewives, car owners, beer consumers, or persons 17 to 21 years of 
age who are not enrolled in college or planning to go to college — 
the recruiting subpopulation. Bass (1969), citing Kuehn and Rohloff, 
states that the problems and difficulty in isolating the effects of 
advertising using the aggregate approach are so formidable that the 
researcher may argue that to make progress in researching the ef fective- 
ness of advertising, research has to turn to the household or individual 
as the unit of investigation. This, then, would be a micro approach. 
Whereas the primary advertising dimension in the aggregate approach is 
a measure of expenditures for advertising, the micro approach emphasizes 
the content of advertising in terms of copy and psychological appeals. 

An example of the latter would be the research performed for the U.S. 

Army Recruiting Command (USAREC) by the Human Resources Research Organ- 
ization (HumRRO) on pretesting Army advertising copy (Wilson & Rosen, 
1973). The contextual variables used in the aggregate approach are 
primarily economic and demographic, such as discretionary income and 
emplo 3 nnent rates. The emphasis in the micro approach would be on socio- 
logical variables relating to the household or psychological variables 
in the case of the individual, such as information search and processing, 
sensory and perceptual processes, emotion and motivation, attitude and 
attitude change, decision making, and personality. Thus, the problem 
of choosing the level of investigation is essentially that of the re- 
searcher. 

The researcher who turns to households and intraindividual re- 
search is in the general area of consumer research, which, in turn, 
involves many scientific disciplines. Reductionism in the subject 
matter of research has not, however, proven to be a panacea for the 
problems that confront researchers in advertising and marketing. For 
example, Jacoby (1976) believes that 85 percent of what had been pub- 
lished as consumer psychology prior to 1968 was of rather low and ques- 
tionable worth and that by 1975, somewhat less than 50 percent of the 
research could be so classified. He attributed the source of less-than- 
satisfactory results, in large part, to a proliferation of inadequately 
researched and highly overlapping measures used in attempting to 
find causal relationships for such central constructs as brand loyalty. 
More recently, methodological and theoretical advances, such as the 
INDSCALE method for scaling individual beliefs and attitudes multi- 
dimensionally (Carroll & Chang, 1970) , and the clear specification of 
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measures and their relationships that could lead to attitude change 
and purchase behavior (Ajzen & Fishbein, 1977; Fishbein & Ajzen, 1972) 
have served to improve research in consumer behavior. Jacoby (1976) 
states that a more comprehensive examination of purchase behavior as a 
complete, individual, information receiving, processing, and decision- 
making procedure has placed consumer research in a better perspective 
for producing reliable and valid results. With respect to the central 
problem of this study — evaluating advertising effectiveness — a frequent 
shortcoming of consumer research is the absence of a demonstration of the 
relevance of research findings to the effective use of advertising. 

To show that certain theoretical concepts can be used to predict a 
limited criterion — such as a certain attitude — at a marginally signifi- 
cant statistical level is a far step from the return that could be 
expected by using the concepts in an advertising campaign. Thus, we 
come full circle to the discussion that initiated this section of the 
study. The individual and aggregate approaches must complement each 
other, and only efforts that are consistent with the best theoretical 
notions in both areas hold promise of being able to evaluate the effec- 
tiveness of advertising in any sales or marketing campaign. 
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NAVY ADVERTISING PROGRAM 



In order to evaluate the effectiveness of Navy advertising, 
there must be a general awareness of the existing organization for 
advertising and its policies and procedures. In addition, specific 
knowledge is required of the objectives of the program. This aware- 
ness or knowledge is required to assess the sources of strengths and 
weaknesses and the effects that factors internal to its operations 
might have on the effectiveness of the advertising. It goes without 
saying that program evaluation is not possible without knowledge of 
its objectives, although there is the collateral area of inquiry that 
could ask what the objectives should be. To explain these aspects of 
Navy advertising creates a problem because of the sheer magnitude of 
detail that is involved and the frequent changes that occur. Accord- 
ingly, the exposition will be selective, rather than comprehensive, 
and the criterion for selection will be those features and items that 
are characteristic of the effort. Attention will be directed to en- 
listed programs. The sources of the material directly pertaining to 
advertising are the fiscal year, communication plans published by 
the Recruiting Advertising Department of the Navy Recruiting Command 
(NRG). Much of the organizational information is based on the research 
done for a systems analysis of Navy recruiting by Arima (1976). 

Organization for Navy Advertising 

The Navy recruiting effort and its advertising are carried out by 
the Navy Recruiting Command (NRG). The headquarters of the organization, 
designated CNRC (for Commander, Navy Recruiting Command), plans and 
directs the advertising program through its Recruiting Advertising 
Department. The operating elements of the command include six Navy 
Recruiting Areas (NRA) and the 43 Navy Recruiting Districts (NRD) 
under them. (See Figure 1.) Both the NRA and NRD manage and use funds 
in the local advertising management system (LAMS) through their public 
affairs officers. The NRDs may have a small laboratory for creating 
graphic materials for local advertising. In addition, each has a large 
store of recruiting aids for distribution to recruiters. Recruiting 
aids include promotional items, such as imprinted pencils; graphic 
items, such as posters and bumper stickers; printed brochures describ- 
ing various Navy programs and Navy life; and counter top or bulletin 
board displays with reply coupons. 

Under the direct operation of CNRC are two, field, advertising 
activities. One is the Navy National Recruiting Information Center 
(NNRIC) at Macon, Georgia, for receiving and processing the toll-free 
telephone Inquiries directed to the wide area telephone service (WATS) 
numbers in response to Navy ads. The processed inquiries are forwarded 
to the other field activity, the Navy Opportunities Information Center 
(NOIC) at Pelham Manor, New York. This office operates the direct-mail 
advertising programs through a contractor. The contractor also 
provides a management information service that stores the information 
developed through NNRIC and the coupons from magazines and direct mail 
enclosures mailed to NOIC. Other information to support the advertising 
effort, much of it generated in the creation of mailing lists, is also 
stored. 
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Figure 1. Geographic Organization of the Navy Recruiting Command 



The Recruiting Advertising Department at CNRC has two divisions 
representing the functional breakdown of the advertising effort. The 
Advertising Operations Division is oriented toward national advertising 
and works through agencies that are contracted to create and market pro- 
grams on a national basis. The primary contractor carries the major 
load for all programs and media. There are also agencies for minority 
programs and the previously mentioned, direct-mail service. The con- 
tracted service provided is essentially of the same general nature as 
that found in the civil sector. The operating link with the contractor 
is through program advertising managers in the Advertising Operations 
Division. Contractors are also represented in the division through 
their liaison personnel and account executives. 

The other main division plans and creates recruiting aids and is 
appropriately named the Recruiting Aids Division. Their work is essen- 
tially done in-house, except for the contracting out of graphic services, 
printing, and similar production activities. To help the division pro- 
vide appropriate materials, there is a RAMS (Recruiting Aids Management 
System) Advisory Council made up of experienced recruiting aids per- 
sonnel in the NRDs. 

There is also a very important office in the Recruiting Adver- 
tising Department — the Management Systems Division — that creates the 
communication plan and tracks and monitors progress in the programs. 

One of the latter is the NALTS (Navy Lead Tracking System) program that 
tracks the disposition of leads that are sent to the recruiter through 
NNRIC and NOIC. This office also monitors the response to individual 
ads and develops a cost-per-lead statistic for every ad. Finally, there 
are offices within the Recruiting Advertising Department to manage the 
distribution of recruiting aids (called RAD items) and to promote public 
service advertising. 

Program Areas 



The Navy advertising programs are closely alined with the recruit- 
ing programs that they support. The first major breakout for recruiting 
is by enlisted and officer categories. Accordingly, there are general 
enlisted and general officer programs. Cutting across both of these areas 
is an ethnicity categorization resulting in minority programs. Enlisted 
programs are further segregated by programs for the reserve, target 
audience, women, and occupational specialty. Officer programs are cate- 
gorized by occupational specialty or source of commissioning, such as the 
medical and NROTC programs, respectively. 

The allocation of advertising funds to major program areas is 
shown in Figure 2, based on the revised FY 1977 communications plan. 
Approximately two-thirds of advertising funds go directly into the 
officer and enlisted programs. The remaining one-third of advertising 
funds go towards advertising support programs and incidental costs 
associated with contractual services, such as labor, overhead, and fees. 
The largest single category of advertising is the general enlisted 
program with 47 percent of the total budget. Of the direct advertising 
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Figure 2 

Allocation of Navy Recruiting Resources 
to Major Program Areas in FY 1977. 
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costs, the ratio of enlisted to officer advertising is 4-to-l. The 
purpose of this research effort is to determine the ef fectivenevs of 
that portion devoted to the enlisted programs. Over the existence of 
the AVF, the enlisted programs have included the following: 



Program 

General enlisted 



Enlisted influentials 



Prior service reenlist- 
ment 

Enlisted retention 



Veteran reserve 



Target 

18-24 year-old males with emphasis on 
18-21 year olds who have graduated 
from high school, are in the upper 
half of this group in general mental 
aptitude, and are mechanically and 
technically oriented. 

Academic teachers, vocational teachers 
guidance counselors, and athletic 
coaches in secondary schools and 
junior colleges. 

Navy veterans in civilian life less 
than 4 years from discharge date. 

Males, 19-25 in ratings E-3 to E-5 
who are high-school educated and 
on active duty. 

Veterans from the Navy and other 
services with transf errable skills. 



Reserves (Active Mariner Primary target: as above for 

and Ready Mariner) general enlisted. Secondary target: 

other enlistment-qualified individuals 
Advertising targeted to both. 

Nuclear f ield/Advanced As above for general enlisted, 

electronics 



The general enlisted program is an umbrella program addressed to 
youth in the 18-to-24 year group that encompasses approaches common to 
all of the enlisted programs. The cornerstone to this approach is a 
unique selling proposition (USP) that distinguishes Navy jobs from those 
provided by the chief competitor, the civilian job market. This unique- 
ness is said to be an extra dimension of adventure, which has been 
emphasized by the specific USP, *'... it’s not just a job, it’s an adven- 
ture.” Special advertising is developed to feature incentive programs. 
As shown above, greatest emphasis is placed on convincing young men in 
the primary target group that the Navy provides jobs and opportunities 
that are superior to their civilian counterparts. 

The influentials program attempts to build a positive attitude 
among the target audience towards the Navy as an attractive job and 
career opportunity so that they will influence 17-to-21 year olds to 
join the Navy. 
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The prior-service reenlistment program is designed to create and 
maintain an awareness and a positive attitude among recently discharged 
Navy personnel of the opportunities for reentering the Navy. 

The enlisted retention program attempts to convince those in the 
target audience completing their first term to remain in the Navy as a 
career choice. 

The veteran reserve program attempts to build an awareness among 
the target population of the opportunity to maintain their link with 
the Navy and maintain and improve their military skills while earning 
Navy pay and benefits. The program is constrained geographically by the 
need for reserve, unit-training opportunities in an area. 

The reserve programs with an active-duty commitment are allocated 
only .15 percent of the monies provided for total enlisted advertising. 
They represent an alternative to a full-term, active-duty commitment and 
do not differ substantially from general enlisted programs. An exception 
is the training and administration of the reserve (TAR) program that 
provides for extended active duty at a reserve activity after a period of 
initial training in the regular Navy. 

Approximately one-half percent of the total enlisted advertising 
budget is spent on the critical, 6-year obligation programs for the nu- 
clear, advanced electronics, and advanced technical fields. These are 
the only enlisted occupational areas having a special advertising pro- 
gram. The advertising is very selective with respect to media and 
copy. 



The Navy advertising program can also be considered from a func- 
tional orientation. In this perspective, most of the foregoing material 
can be considered as national media advertising. Other functional cate- 
gories include the RAD aids program, the local advertising management 
system (LAMS), and public service advertising. 

The RAD aids program attempts to provide materials to recruiters 
that can be used at the local level as an extension of CNRC’s national 
advertising campaign. To complement the national advertising, RAD aids 
maintain common copy features and the overall concept of the national 
umbrella campaign. Since printed items are purchased or produced in 
large quantity, many of the items may have the USP of previous campaigns. 
Because most of the major aids are designed for the general enlisted 
program, they cover areas included in all of the enlistment programs. 
Nevertheless, aids may be produced to highlight special programs or for 
emphasis. For example, the booklet, ”Navy Training - Civilian Careers,” 
provides a description of the Navy’s occupations and relates them directly 
to the Labor Department’s "Dictionary of Occupational Titles" for the 
benefit of vocational counselors. The booklet, "Navy Campus for Achieve- 
ment," highlights educational and training opportunities for both officers 
and enlisted persons. The most popular booklet gives an attractive por- 
trayal of Navy life and opportunities and is requisitioned at a rate of 
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over 100,000 copies per month by the field (Arima, 1978a), Included 
among RAD items are a variety of canned or taped radio and TV spots for 
public service advertising. The cost of recruiting aids is shown by 
quarters for FY 1976, 1977, and the temporary quarter TQ 1976, in 
Table 3. 

The local advertising, or LAMS, program is designed to permit 
operators and canvassers in the field to capitalize on local opportun- 
ities for publicizing Navy opportunities. In the past, a portion of the 
advertising budget was earmarked for the LAMS program, and the funds were 
provided the NRA commanders who further allocated monies to their NRDs 
according to demonstrated requirements. More recently, NRDs have been 
required to propose an advertising plan, which is reviewed by the NRAs 
and at the CNRC level before approval and funding. Typically, the money 
has been spent on advertising in school newspapers, local sports and 
rodeo programs, telephone yellow pages, mail outs from local recruitiers, 
van-mounted mobile display units, taxi -mounted displays, and so forth. 
Expenditures for LAMS are shown by quarter in Table 3, also. 

Finally, there is the public service advertising program, the 
objectives of which are identical to the paid advertising campaign. At 
one time, this was the sole source for radio and TV advertising, owing 
to the Congressional prohibition previously mentioned. Magazines, news- 
papers, outdoor, and public transit advertising are also solicited. All 
materials used in the public service program are produced by other activ- 
ities; the Public Service Division’s primary mission is to solicit and 
maintain lists of media representatives who will provide advertising 
space or time on a continuing basis. The emphasis placed in the solici- 
tation is that the media would be performing a public service by inform- 
ing the public of opportunities in the Navy. 

Media Utilization 

The pattern of utilization of media in the Navy’s advertising 
program during the AVF years has been conditioned by constraints on the 
use of the electronic media into the following three periods: 



During the first period, media advertising was in magazines, direct mail, 
outdoor, newspapers, and supplements. Supplements were used in only the 
second and third quarters of 1976. The inclusion of radio and television 
changed the distribution of resources among the quarters and among media. 

Previous to radio and television, advertising was concentrated in 
the second and third quarters. After the electronic media became 
available, advertising resources have been concentrated in the second and 
fourth quarters. The general policy is that peak advertising should occur 



Media Constraint 



Period (Calendar Years) 



No radio or TV 
Radio, no TV 
Radio & TV 



July 1973 to Third Quarter 1976 

Fourth Quarter 1976 to First Quarter 1977 

Second Quarter 1977 to present 
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Table 3 



Navy Expenditures for Local Advertising (LAMS) 
and Recruiting Aids (RAD) 



Year 






Quarter 




Total 




1 


2 


3 


4 










LAMS 






1976 


186,405 


426,713 


302,799 


206,403 


1,122,320 


1977 


260,360 


302,520 


329,100 


441,649 


1,333,629 








RAD 






1976 


(N.A.) 


2,894,921* 


1,065,084 


2,328,031 


6,288,036 


1977 


2,471,120 


4,658,691 


820,212 


802,164 


8,752,187 



*RAD expenditures for the first quarter of 1976 were not available 
(N.A.)* The second quarter, however, includes both the first and 
second quarters. 
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in the periods April to May and September to November on the premise 
that the decision time of the primary target occurs prior to graduation 
in the late spring and mid winter. Advertising for the small retention 
program is peaked during June, July, and August prior to the beginning 
of the school year and family moves and decisions are made. The distri- 
bution of resources in time is also affected by the characteristics of 
the media. Magazine advertising has a relatively long effect, since it 
is hard copy that is retained. Coupons from magazine ads continue to 
come in for many weeks following the insertion of the ad, and it is not 
unusual for some to arrive several months later. On the other hand, time 
for network and syndicated television programs must be purchased well in 
advance. Obviously, in contrast to hard copy, the life of one presenta- 
tion on TV is ephemeral. Thus, large-scale utilization of television 
tends to structure advertising, and it must be programmed well in ad^ 
vance. 



The distribution of resources over media is shown in Table 4 by 
quarters selected for their comparability, proximity in time, and 
representativeness of the three periods in media utilization. To ex- 
pand on the relative amounts obligated to the use of the media shown in 
the percentages of Table 4, policies and practices with respect to the 
utilization of individual media will be noted. Network and syndicated 
television is used as a secondary medium to capitalize on the sight, 
sound, and motion that the medium provides to convey the advernture 
theme. Other considerations are its broad reach and increased exposure 
to inf luentials . 

Paid radio is utilized because of its fast reach and frequency of 
impact. It provides flexibility, a minimum lead time, and considerable 
target selectivity. For example, the veteran reserve program can be 
limited to the urban centers where recruiting for the program occurs. 

The Armed Forces Radio-Television Service (AFRTS) is used in overseas 
areas, especially for the retention program. 

Magazines provide an opportunity for a continuous, 12-month 
program that transmits detailed copy. As stated above, magazines are 
hard copy and durable. Magazines also provide reasonable target selec- 
tivity — e.g., counselors through professional journals — and lead gener- 
ation by inserted pop-up cards or on-page coupons. The latter cost much 
less, and some even believe that the effort and cost required to cut out 
and mail the coupon is a screen for the overly casual or crank respon- 
dent. 

Direct mail is an especially versatile medium because it permits 
the most specificity and segmentation of the target audience — e.g., 
this year’s graduating seniors in predominantly black high schools. 
Direct mail also has a good lead generation capability through mail-back 
postcards, any desired amount of detail can be presented (although it 
may not be read), and it is used in an intrinsically action-oriented 
approach, rather than the awareness or attitude orientation of other 
media. The veteran reenlistment program is carried out with direct 
mail as the major vehicle. It is also an important medium for the 
retention program. 
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Table 4 



Distribution of Media Advertising Dollars with and 



without Radio and Television 



Media 


3rd Quarter 
1976 


1st Quarter 
1977 


4th Quarter 
1977 


Television 








898,328 








(47) 


Radio 





343,993 


591,807 






(34) 


(29) 


Magazine 


808,460 


463,497 


161,957 




(55) 


(46) 


(09) 


Direct Mail 


32,256 


143,357 


290,309 




(02) 


(14) 


(15) 


Outdoor 


579,488 


31,462 


— 




(40) 


(03) 




Newspapers 


3,202 


20,883 


— 




(0.2) 


(02) 




Supplements 


41,624 


— 


— 




(03) 






All Media 


1,465,030 


1,003,192 


1,912,401 




(100)* 


(100) 


(100) 



*Numbers in parentheses are percent of total media expenditures for 
the quarter. They may not sum to 100 due to rounding. 
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Newspapers and supplements are used sparingly at the national 
level. Navy base papers and the Navy Times are used in the reenlist- 
ment program, and metropolitan dailies are used in selected areas for 
the veteran’s reserve program. In other cases, newspapers may be used 
to try to rectify shortages in critical occupational areas. 

Outdoor advertising is used fairly extensively, as Table 4 shows, 
for rather limited purposes. While outdoor advertising has temporal 
stability, its problem is unique in that the viewer is usually in rapid 
motion. Accordingly, very little information can be transmitted. In 
the Annheuser-Busch studies, for example, outdoor advertising was 
completely curtailed when it was learned that the viewing time was just 
a matter of seconds and that the average person was exposed to the words, 
Annheuser-Busch, some 10 or 11 times a day. There was no basis, it was 
reasoned, for adding another similar occasion to an already very frequent 
event. In the same vein, the Navy uses outdoor advertising as a support- 
ing medium in top Navy recruiting markets to reinforce the message of 
the USP. Outdoor advertising can provide broad coverage, and it can be 
used in a reminder role, such as on or near Navy bases as a link to the 
career counselor in the reenlistment or retention program. Its use in 
the retention program and the advantage of good weather make outdoor 
advertising a mid-year project. Its area specificity also makes it 
useful for the veteran’s reserve program. 

Objectives of Navy Advertising 

The communications plans of the Recruiting Advertising Department, 
in presenting the various programs, relates each to the marketing objec- 
tives for the program. There is the implicit intent that the advertis- 
ing program should contribute toward the attainment of the marketing 
objectives, since advertising is a part of the marketing plan. In addi- 
tion, the communications plans list advertising objectives separately 
for the various programs. Inspection of the marketing and advertising 
objectives over the years suggests that there is not always a clear 
distinction between the two. In one plan, "deliver stronger, more 
motivating copy" is listed as a marketing objective, and "build a 
stronger motivation toward Navy enlistment" is listed as an advertising 
objective, while in a different plan, "establish long-term positive 
attitudes towards the Navy" is included as a marketing objective. In one 
plan, increasing traffic at the recruiter level is listed as a marketing 
objective, and in another plan, it is listed as an advertising objective. 
In addition to the marketing and advertising objectives, other objectives 
slip into the communications plan through sections such as those listing 
creative objectives and strategies. It seems, however, that some overall 
objectives of Navy recruiting advertising are to have a positive effect 
on the attainment of recruiting goals, create a long-term positive atti- 
tude toward the Navy, and to increase traffic at the recruiting-station 
level. Another that might be included in this overall hierarchy is to 
generate leads. 
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The advertising objectives listed in the revised FY 1977 communi- 
cations plan for the general enlisted program are quite comprehensive 
and representative of listings in other plans and sections of plans. 

The listing is preceded by a statement that the objectives are to be 
achieved by building a greater awareness of, and positive attitudes 
toward, the Navy. Whether either or both are necessary or sufficient 
for attaining the objectives is, as explained previously, open to ques- 
tion and a matter for research. The list of specific objectives follows 

1. Greater walk-in traffic at the recruiting station. 

2. More favorable response to a recruiter’s telephone call. 

3. Increased calls to the toll free number. 

4. More leads generated to NOIC. 

5. Greater discussion among inf luentials . 

6. More favorable reception to recruiter’s request to visit 
groups . 

7. Increased dialogue between prospect and recruiter. 

8. Better understanding of Navy benefits. 

9. Greater receptivity to direct mail solicitations. 

10. Greater respect for the Navy recruiter within the community. 

11. Increased self-esteem among the field recruiters. 

12. The understanding by the prospect that jobs and careers in 
the Navy are more rewarding than most civilian jobs. 

The advertising objectives for the special programs are generally 
to create awareness of and a favorable attitude toward the programs. 

In terms of the discussion on problems in evaluating advertising, the 
advertising objectives as stated are the presumed mediators of the ul- 
timate objective — attaining the required numbers and quality of recruits 
Even the action objectives, such as walking into a recruiting station, 
calling the toll-free number, or sending a coupon to NOIC, are one im- 
portant step short of the desired action of signing a enlistment con- 
tract. 
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PROBLEMS IN EVALUATING THE EFFECTIVENESS 
OF NAVY RECRUITING ADVERTISING 



The problems that will be faced by the researcher or analyst who 
attempts to evaluate the effectiveness of Navy advertising will dupli- 
cate many of the. points made previously with respect to advertising in 
general. Only the setting and specific issues may be different. There 
will be the overriding problem caused by the multiple determination of 
recruiting production, the need to choose among methodological approaches 
and levels of evaluation, and limitations in the available or collectible 
data. 

Multiple Determinants of Personnel Acquisition 

The determinants of production by the recruiting force — i.e., the 
bottom line — are many. They can be divided into sources that are 
associated with the recruiting and personnel procurement effort, itself, 
and those generated by temporal and environmental factors outside the 
control of the recruiting effort. These will be referred to as internal 
and external factors in the sense that economists use the labels endogen- 
ous and exogenous sources. When such a complex phenomenon as recruiting 
production is categorized by two source variables, the interaction of 
the two must also be considered as another possible source. That is, 
the effect on recruiting of one of the endogenous variables might depend 
on the presence or absence of one of the exogenous variables. 

Internal Determinants of Productivity 



As plans mediated the levels of both sales and advertising in 
civilian enterprise, goals will be found to perform a similar role in 
the Navy’s recruiting advertising. Thus, accessions, rather than re- 
flecting advertising, may be dictated by goals. During a period when 
goals act as a lid on recruiting, the number of recruits attained will 
be a direct result of the goal, and any amount of advertising will not 
increase numbers of persons being recruited. Using a Marine Corps 
example, knowledge of recruiter man months at a recruiting station pre- 
dicted recruiting station output with a correlation in excess of .99 
(Sullivan, 1976), since goaling was also directly related to available 
recruiter strength. When the system is in equilibrium, goals, a popu- 
lation parameter, recruiter strength, and recruiter production should 
predict each other equally well and to a high degree, leaving little 
room for the effects of other factors. 

Changes in qualitative requirements may also have a profound 
effect on production. For example, towards the end of the first year 
of the AVF (FY 1974), goals were increased sharply, but production in 
total numbers was attained by a drop in the quality of recruits (Arima, 
1976). On the other hand, the requirements for high school graduates 
was raised at the start of FY 1978 as a means of controlling the attri- 
tion rate, since the probability of a diploma, high-school graduate 
finishing a term of service is considerably better than that of a 
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hlgh-school dropout. After the new requirement, however, production 
declined in succeeding months, sometimes at a dramatic rate. 

For whatever reason, the Navy Recruiting District was found by 
Arima (1978b) to be a significant determinant of recruiter productivity, 
and Brown, Wood, and Harris (1975) found that the one best predictor of 
Army recruiter productivity was the average rate for the recruiter’s 
district. As the basic management unit in the field, this finding 
regarding the districts probably reflects a combination of management 
prerogatives and practices, in addition to external factors. For ex- 
ample, the assignment of resources and goals to districts may not be 
equitable, considering territorial factors conductive to recruiting. 

It may also reflect the capabilities of the staff, and especially those 
of the commanding officer of the NRD. 

Finally, recruiter effort and capability are also Important 
factors in determining production. Arima (1978b) found that the re- 
cruiter accounted for some 35 to 40 percent of the variability in pro- 
duction among recruiters. When aggregate measures of productivity are 
used, such as for the station or NRD, the individual recruiter is no 
longer a variable in the analysis but contributes significantly to 
noise in the data. 

In addition to these factors directly associated with the re- 
cruiting effort, there are other activities that form a part of the over- 
all marketing plan. Under the aegis of the Advertising Support Depart- 
ment of CNRC, many promotional activities are sponsored or carried out. 
These Include the Sea Power lecture series, the Blue Angels flying team, 
solicitation of the support of local Navy-related organizations such as 
the Navy League, sea tours, sponsored tours to Navy installations, and 
an educational liaison program. Close coordination and integration of 
these activities exist with the Navy’s public information program. 

Incentives in the form of choice of coast, buddy enlistment op- 
tions, the enlistment of an entire local group to be trained together 
initially, and other incentives are created from time to time. Indi- 
viduals who are in the delayed enlistment program (DEP) and waiting 
for their active duty entry may be advanced one grade for inducing 
friends to enlist, thus giving the person more money from the first day 
of service. A complicating factor is the use of advertising to publi- 
cize these opportunities. How much of the effect, if any, is due to 
the specific incentive and how much to the advertising? 

The enlistment programs or options — the items being marketed — 
are also important determinants of enlistment decisions. As of this 
writing, the choice fields Involving high levels of training — advanced 
electronics, nuclear, medical technology, advanced technical field — 
require a six-year obligation. If they were cut to four, what would 
be the difference? At one time, there was a three-year enlistment 
option, now the shortest enlistment (excluding reserve categories) 
is for four years. Enlistment options can be configured in other ways 
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than by length of service obligations. All of these programs are 
heavily advertised — for example, under the overall Navy Campus for 
Achievement (NCFA) program (Arima, 1978a). Some part of the success 
of the programs, if any, must be due to the nature of the options 
offered and also the manner in which they are advertised. 

These and other aspects of the recruiting, marketing, or 
staffing program — generally termed personnel acquisition in the Navy — 
all contribute to an individual’s decision to enlist. To isolate the 
effects of advertising from them, is a formidable effort. But this is 
merely a listing of the internal factors. 

External Determinants of Productivity 

There are many external factors that have been examined for 
various reasons. The distribution and allocation of recruiter and 
other recruiting resources, the goaling of recruiting units, and the 
content and targeting of advertising and promotions may be some of 
the purposes for examining them. Depending on the methods of exam- 
ination, the time period in which the examination took place, the 
particular sample, and the level of examination — individual or aggre- 
gated, some factors have been found to have a significant effect in 
one study and then no effect in others. Rather than reviewing this 
extensive literature in detail, typical examples will be listed. 
Economic variables, such as emplo 3 mient opportunities, family income, 
and wage rates in an area, have been given much consideration. Demo- 
graphic and social variables have also been used, such as workforce 
composition, age distribution of the population, ethnic profile of 
a community, and population density. The impact of a military pres- 
ence (bases) in an area has been hypothesized as having both facili- 
tative and detrimental effects on recruiting. The competition of 
recruiting advertising by the other services may be an important 
factor. Educational characteristics are another dimension and have 
included the presence or absence of different types of institutions, 
proportion of students who finish high school, quality and availabil- 
ity of vocational education, pattern of continuation into post-high- 
school educational paths, and the position of an area on standard- 
ized tests. Finally, there has been a large number of studies trying 
to identify the attitudes, desires, and propensities of individuals 
for enlisting in the military services. Any one of these factors or 
any combination thereof .may have differential effects on the success 
of recruiting in a particular area. 

One way to handle this seemingly endless list of potential 
sources of effects on recruiting productivity is to ignore them and 
treat them as random variables. Obviously, this would then necessitate 
large samples in order to provide enough degrees of freedom for a 
small advertising effect to be detected. One potential source of 
problems due to the numerosity of factors is their colinearity, but 
this can also be an advantage that can be exploited by reducing them 
to a few underlying factors. These considerations, however, are 
more appropriate to a discussion of methodological approaches and 
data requirements. 
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Methodological Considerations 



Perhaps a highlight of the continuing effort to develop means 
for evaluating Navy advertising effectiveness was a one-day seminar 
on ’’recruiting resources allocation and recruiting advertising 
impact” held in Alexandria, Virginia, on 30 January 1978 under the 
sponsorship of the Office of Naval Research. Since the leading pro- 
ponents of the various approaches to evaluating advertising effective- 
ness were represented at the seminar, each of the approaches received 
its share of support. In general, the recommendations included those 
for controled experiments, the regression approach, modeling and 
optimizing the recruiting and advertising processes, and individual 
consumer behavior research. No consensus was reached or really 
sought. The problems and possible approaches were aired, and they 
repeated the arguments in the civil sector. Thus, the approach that 
will be taken will depend on the orientation of the researcher, and 
no one method has to be chosen to the exclusion of other approaches. 

In view of the Navy’s expressed desires for effectiveness research, 
there was emphasis on means for evaluating the role of advertising 
with respect to recruiting production, rather than some intermediate 
criterion. 

Accordingly, this study effort will adopt a regression approach 
for several reasons. First, because it is exploratory, it would 
appear sensible to examine the situation with available historical data. 
This approach would help to Identify the variables that affect re- 
cruiting production and may even provide some indication of advertising’s 
effects on production. Control experiments could profit greatly from 
such an exploratory study with respect to the variables that must be 
controlled, randomized, or parameterized; the levels of the experi- 
mental, expenditure variable to be used; and the appropriate duration 
of the experiment. In addition to these reasons, difficulties in 
executing a controlled experiment are greater than those for a re- 
gression study. These difficulties lie in the potential reactivity — 
and resulting bias — of the recruiting organization and personnel to 
the experimental manipulations, the need to control the influence 
of several advertising media that operate simultaneously, the need to 
obtain other-service cooperation to control for service advertising 
levels in any area, and the need to continue the experiment for a 
sufficiently lengthy period to permit stabilization of production at 
the experimental support levels. 



Data Availability 

Given the decision to evaluate the Navy’s advertising effec- 
tiveness using currently available data, it was necessary to determine 
what data were available. Generally, the immediately available ad- 
vertising data were at a relatively high level of aggregation. Ac- 
cordingly, while it was possible to create production files using 
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information at the individual recruit level from the Enlisted Master 
Record (EMR) , a more aggregated source of production was sought. Un- 
fortunately, almost all recruiting production data relate to shipments 
to the Recruit Training Centers (RTC) . This, of course, is the ulti- 
mate bottom line of the recruiting system, but it is a poor indicator 
of enlistments because the individuals shipped could have enlisted 
at any time up to one year prior to the shipment date under the delayed 
entry program (DEP) . A recruiting production category reflecting the 
enlistment data is contracts signed. This information was available 
beginning in July 1976 in terms of the average number of new contracts 
signed per canvasser per month by NRD. Accordingly, references hence- 
forth to "contracts signed" will be to this data source. 

Data relating to national advertising programs were available in 
several forms. The first category was a record of leads generated by 
coupons and the WATS numbers. A running record of leads received is 
kept for each ad insertion, and a cost-per-lead datum is available 
for each ad. A lead trackings system (NALTS) monitors for 60 days the 
outcomes of a sample of leads that have been referred to recruiting 
stations. These data more-or-less speak for themselves. They could 
be important for evaluating copy, consumer segments, and publications 
(individually and by classes). However, they were not considered to 
be useful for the analyses contemplated. 

Data for national advertising were also available in the form of 
total expenditures per month at a national level by media and major 
program category — e.g., general enlisted. These data were aggregated 
at such a high level that they could not be used to evaluate advertis- 
ing effects at an operational level of recruiting, such as the NRD. In 
addition, evaluation of advertising at an aggregated, national level 
would be difficult because of the simultaneity problem involving mana- 
gerial decisions on the seasonality of recruiting and the shipping 
schedules based on projected training facilities. That is, the train- 
ing pipeline determines the flow, variations in the flow depend on 
ideas of the seasonality of recruiting, and both advertising and pro- 
duction are determined by these considerations. Any relationship found 
between advertising and production under these conditions would tend 
to be spurious. Considerations such as these made it unlikely that 
the national advertising data could be used to any advantage. 

Expenditures for the LAMS program and dollar equivalents of RAD 
items requisitioned were available at the NRD level aggregated by 
quarter. 

This picture of available data was not encouraging. An attempt 
was made to relate production at the NRD level to the available adver- 
tising data with highly indeterminate results. The available data 
were supplemented with self-report data available from the twice-yearly , 
youth attitude tracking studies conducted for the Department of Defense 
by Market Facts, Inc. An analysis of these data confirmed the importance 
of attitudes and certain demographic variables on the professed propensity 
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to enlist and the reported, self -initiated contacts with Navy recruiters 
(Rhodes Associates, 1978). They were inconclusive with respect to the 
effects of advertising awareness measures or the limited data on LAMS 
and RADS expenditures (Table 3) • 

The limitations in the data were brought to the attention of the 
Navy Recruiting Command, and arguments were made for more detailed data 
showing the distribution of national media advertising. The Recruiting 
Command then let a contract with its agency, Ted Bates, Inc., to break 
out all national advertising in several program categories to the 
county level by month for calendar years 1976 and 1977. Completion 
of this work effort made possible the analysis in the second part of 
this report. 
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PART II, EMPIRICAL STUDY 



THE RELATIONSHIP BETWEEN ADVERTISING RATES 
AND ENLISTMENT RATES IN THE NAVY 
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METHOD 



Approach 

In order to take advantage of the allocation of national media 
advertising expenditures to counties in the study by Ted Bates (1978), 
the county within the continental United States was taken as the unit 
for analysis. This required the reallocation of quarterly LAMS and 
RAD expenditures at the NRD level to counties in monthly increments. 
When this was done, information representing all Navy advertising 
expenditures was available for each of over 3,000 counties by month 
for 1976 and 1977. The advertising data were then supplemented by 
variables reflecting management practices, such as the allocation of 
canvassers to counties, and the population characteristics of the 
county that were thought to have the most impact on the production of 
recruits. Finally, based on the record of all individual accessions 
to the Navy in 1976 and 1977, the number of new, first-term, male 
recruits was determined for each county by month. Using the recruiting 
data as the dependent variable, an ordinary least squares (OLS) , 
multiple regression model was fitted to the data. Advertising lagged 
by one and two months, as well as current advertising, was used in 
the analysis. The details of this approach are explicated in the 
material that follows. 

Data Sources and Basic Study Variables 

Advertising Data 



The Ted Bates agency provided data on the Navy’s expenditures for 
national media advertising programs by county and month for calendar 
years 1976 and 1977. Only data for the general enlisted program 
were used in this study. The media included television, radio, maga- 
zines, direct mail, outdoor, newspapers, and supplements. As pre- 
viously stated, paid radio advertising did not begin until November 
1976 and television, not until April 1977 because of Congressional 
prohibitions against the use of paid advertising in the electronic 
media. The national, monthly distribution of advertising expenditures 
over the media is shown in Table 5. 

The television dollars per county for a particular program were 
calculated by first using A. C. Nielsen impressions per station for 
men 18-34 years old, determining the proportion of all impressions 
achieved by each station, and prorating the costs of the program to 
each station on this basis. The station dollars were then assigned 
to counties in proportion to each county’s average viewing hours of the 
station based on periodic Nielsen county-coverage studies. The county 
program dollars were then summed by month for each county. 

County dollars for network radio programs were determined in a 
similar manner using ARB (Advertising Research Bureau) impressions per 
station and distributing station dollars to counties using the Pulse, 
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MONTHLY MEDIA EXPENDITURES* 
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^Excludes newspaper and supplements, which were not a part of the general enlisted program 



male listening audience figures for stations by county. Program 
dollars for new stations not in the Pulse calculations were attributed 
directly to the county in which the station was located. This 
procedure was also used for spot broadcasts. 

The ABC (Audited Board of Circulation) state and county data 
bank of audited circulations was used to assign magazine dollars to 
counties in 52 percent of magazine advertising. The other 48 percent 
was estimated from state circulation data provided by ABC or obtained 
directly from publishers and then prorated to counties on the basis 
of 1977 population statistics provided by Sales Management, Inc. 

Dollars were distributed directly into the county of origin in limited- 
distribution advertising situations, such as programs for special 
events. 

Dollars for outdoor advertising were taken from the VCS (Visual 
Campaign Summary) market data and prorated to counties on the basis of 
QMA (Qualified Male Availables)^ in the counties within each VCS 
market summary. 

Direct mail dollars were estimated by using a typical mailout 
in 1977 to determine the number of mailouts by zip code. A tape 
provided by Zip-O-Data was used to assign the zip-code data to 
counties. The exact dollars by county for each mailout were then 
obtained from the direct-mail distributor. 

Newspaper and supplement advertising was assigned to counties 
using ABC data or circulation information obtained directly from 
publishers in a manner similar to magazine advertising. 

In order to reallocate the LAMS and RAD expenditures which were 
provided by the Navy Recruiting Command for the 43 NRDs by quarter, 
it was first necessary to track the fluctuating mapping of counties 
to NRDs. This was done for the entire period from January 1976 through 
December 1977. The NRD dollars for LAMS and RAD were then assigned 
to the constitutent counties in proportion to their QMA. The quarterly 
amounts were then divided equally into the three months making up the 
quarter, except for the first six months of 1976, when they were 
divided into six equal, monthly statements. 

The foregoing operations provided an assignment of all Navy 
advertising dollars for the general enlisted recruiting program to 
each of 3,074 counties over a two-year period by month. 

Recruiting Data 



The key recruiting variables were those for goals, accessions, 
canvassers, and the county population base. Goals were for first- 
term, male (QUEBEC or Q) enlistees. The population base was a 



^The QMA is discussed below in conjunction with the description 
of recruiting data used in this study. 
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second (1977) revision of the county-level QMA which was provided by 
the Defense Manpower Data Center (DMDC) . ^ The QMA was constant for 
all 24 months of the study. Goals and canvassers were obtained for 
each NRD from the monthly recruiting sunmaries of the Navy Recruiting 
Comnand. They were prorated to counties within each NRD on the basis 
of the QMA. National monthly goals and canvasser strengths are shown 
in Table 6. 

Appropriate accession figures proved to be difficult to obtain, 
since the usual accession figures reflected shipments to the Recruit 
Training Centers (RTC) . While such accession figures are the bottom 
line for the recruiting system and of the utmost importance to its 
management, they are inappropriate to use as enlistment figures for 
research in recruiting because those being shipped could have actually 
enlisted up to one year previously in the delayed entry program (DEP) , 
where the majority of shipments to the RTCs originate. What was 
needed was a ^'contract signed’* date to identify the act of enlisting. 
This was obtained from the individual, enlisted master record (EMR) 
tapes by DMDC using the pay entry base data (PEBD) , appropriately 
edited. All first-term, male enlistees with pay entry base dates in 
1976 and 1977 were identified by date of enlistment, county of 
origin, number of years of education completed (high school graduation), 
and AFQT (Armed Forces Qualification Test) score. They were then 
aggregated by county and month according to total Q enlistments, enlist- 
ments of high school graduates, and enlistments of recruits who were 
high school graduates and had AFQT scores in the upper half of the 
AFQT distribution.^ National monthly enlistments in these categories 
are shown in Table 7. 

Situational Data 



The number of high school graduates in each county for 1976 
was obtained from DMDC and used as a measure to partially validate the 
revised QMA statistics and also to provide an indicator of the quality 
potential of a county, since high school graduation is one of the two 
most important screening variables for Navy recruiting. The 1976 
figure was used as a constant for the entire 24 months of the study 
period. 



2 

The QMA data were originally compiled for each county using the 
1970 census of males aged 17-24 years, inclusive. This figure, the 
military availables (MA) , was adjusted for each county on the basis of 
empirical qualification rates for general, military service to result 
in qualified military availables (QMA). It was revised in 1976 on the 
basis of birth and death rates, population migration statistics, and 
more current qualification data. It was again revised in 1977 to 
reflect more accurate acceptance rates. 

The category of enlistees who were high school graduates and 
had AFQT scores at or above the 50th percentile will be identified as 
GRAD50 enlistees and may be referred to as ’’quality” enlistees. ’’Total 
enlistments” refers to first-term male enlistees and may be referred 
to as QUEBEC or ”Q” enlistees. 
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Table 6 



MONTHLY Q GOAL, CANVASSERS, AND UNEMPLOYMENT 



YEAR 


MONTH 


Q GOAL 


CANVASSERS 


UNEMPLOYMENT 

PERCENTAGE 


1976 


Jan. 


5,953 


3,241 


19.9 




Feb. 


4,943 


3,236 


19.2 




March 


4,631 


3,215 


19.1 




April 


4,293 


3,178 


19.2 




May 


4,935 


3.147 


18.5 




June 


7,765 


3,158 


18.4 




July 


9,500 


3,179 


18.1 




Aug. 


9,500 


3,236 


19.7 




Sept. 


9,500 


3,266 


18.6 




Oct. 


7,992 


3,221 


19.0 




Nov. 


6,216 


3,247 


19.0 




Dec . 


4,688 


3,222 


18.9 


1976 


Total 


79,916 


3,212* 


19.0* 



1977 


Jan. 


7,249 


3,229 


18.7 




Feb. 


5,302 


3,258 


18.5 




March 


4,775 


3,266 


18.8 




April 


4,275 


3,303 


17.8 




May 


4,775 


3,336 


17.9 




June 


8,552 


3,362 


18.6 




July 


9,795 


3,382 


17.4 




Aug. 


11,442 


3,495 


17.5 




Sept. 


11,083 


3,477 


18.1 




Oct . 


6,375 


3,420 


17.3 




Nov. 


5,061 


3,392 


17.1 




Dec. 


3,836 


3,395 


15.4 


1977 


Total 


82,520 


3,360* 


17.8* 




*Yearly totals for these 
preceding 12 months. 


categories 


are averages ol 
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Table 7 





MONTHLY 


FIRST-TERM (Q) 


ENLISTMENTS 




YEAR 


MONTH 


TOTAL 


HIGH SCHOOL 


GRADUATES 






ENLISTMENTS 


GRADUATES 


ABOVE 50TH % 


1976 


Jan. 


9,021 


5,834 


4,175 




Feb. 


7,897 


5,114 


3,765 




March 


8,138 


5,612 


4,218 




April 


6,666 


4,650 


3,471 




May 


5,834 


4,209 


3,131 




June 


7,067 


5,553 


3,813 




July 


7,531 


5,691 


4,173 




Aug. 


7,761 


5,886 


4,346 




Sept. 


8,207 


5,927 


3,991 




Oct. 


7,891 


5,459 


3,612 




Nov. 


9,098 


6,132 


4,449 




Dec. 


16,457 


11,291 


8,147 


1976 


Total 


101,568 


71,358 


51,309 



1977 


Jan. 


7,025 


4,703 


3,194 




Feb. 


7,191 


4,651 


3,172 




March 


7,764 


4,990 


3,489 




April 


6,332 


4,073 


2,886 




May 


5,900 


3,818 


2,738 




June 


7,633 


5,154 


3,461 




July 


7,852 


5,213 


3,483 




Aug. 


8,677 


5,616 


3,809 




Sept. 


7,443 


4,374 


2,989 




Oct. 


5,951 


3,741 


2,651 




Nov. 


6,927 


4,033 


2,904 




Dec . 


6,861 


3,836 


2,719 


1977 


Total 


85,556 


54,202 


37,495 
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The Department of Labor’ s national , monthly unemployment rates 
for youths aged 16-19 years were used as a gross indicator of unemploy- 
ment trends and assigned to all counties. These rates are shown in 
Table 6. 

Data Summary 

In summary, the basic unit for this study was the county. All 
expenditures for advertising programs by media, important recruiting 
management and production data, and two qualitative indicators — high 
school graduates and unemployment rate — were obtained or calculated 
for each county. Where the available data were not directly assign- 
able to counties, an appropriate and available basis was used to prorate 
the data to the counties. This was done for each month in the two-year 
period and resulted in a total of 73,776 counties-by-month cases. 

Data Refinement for Analysis 

Outliers 



There were three sources of outliers in the data. The first of 
these can be seen by an examination of Table 7, where the enlistments 
for December 1976 are more than twice that for December 1977 and for 
the average of the preceding 11 months of 1976. This phenomenon was 
a manifestation of the cessation of Veteran’s benefits for persons 
enlisting after the end of December 1976. This event probably influ- 
enced some enlistments in November 1976 as well, but the effect is 
not as pronounced. For example, enlistments in November 1976 were 
approximately equal to those in January 1976. Accordingly, only data 
pertaining to December 1976 were eliminated from the analysis. 

The second and third sources of outliers were due to the small- 
ness of some counties. This resulted in many county-month cases 
where there were no enlistments. The effect was greater for the 
quality (GRAD50) enlistees. This situation could be expected with 
3,000 counties and a monthly Q enlistment rate of less than 8,000 for 
the entire recruiting command during the two-year period. Leaving in 
the county-month cases with zero enlistments resulted in distributions 
that were excessively peaked and bimodal. Accordingly, zero-enlistment 
county-months were deleted from the analysis of both total enlistments 
and the GRAD50 enlistments. 

When, as will be explained later, the enlistment data were 
normalized for the county QMA, the distribution showed a lengthy, posi- 
tive tail of cases with exceptionally high rates. Again, it was 
apparent that these cases were also due to the smallness of the counties. 
That is, when a very small county experienced several enlistments in a 
single month, the rate was exceptionally high. A cummulative frequency 
distribution of total monthly enlistments per QMA is provided in Table 8. 
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Table 8 



DISTRIBUTION OF TOTAL MONTHLY ENLISTMENTS PER QMA 



INTERVAL 


FREQUENCY 


CUMULATIVE FREQUENCY 


Less than .0005 


7,973 


23,6 


,0005 to .0010 


11,936 


58.8 


,0010 to .0015 


6,083 


76.8 


.0015 to ,0020 


2,979 


85.6 


.0020 to .0025 


1,586 


90,3 


.0025 to .0030 


1,065 


93.4 


.0030 to .0035 


649 


95.3 


.0035 to ,0040 


402 


96.5 


.0040 to .0045 


281 


97,4 


.0045 to .0050 


221 


98.0 


.0050 to .0055 


158 


98.5 


.0055 to .0060 


116 


98.8 


Greater than .0060 


408 


100.0 
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The 682 county-month cases where the rate exceeded .005 enlistments 
per QMA were deleted from the analysis as outliers. These 682 cases 
represented two percent of the cases that had enlistments during a 
given month. 

As a result of the foregoing operations, of the original 73,766 
county-month cases, 33,175 remained for the total enlistments category, 
and 22,844 remained for the GRAD50 quality enlistments category. Thus, 
the analyses that will be conducted are for the rate of enlisting, 
given that enlistments have taken place. The question remains as to 
whether there are other characteristics that distinguish the deleted 
county-month cases, other than their smallness. A comparison of the 
two outlier categories with the cases included in the study is made in 
Table 9 on important study variables. It is very apparent that small- 
ness is the characteristic that distinguishes the outliers from the 
included cases, especially with respect to the excessively high rates 
of enlisting. All of the other variables, with the possible exception 
of national media advertising, are essentially identical for all 
categories. The national media rates are the smallest for the zero- 
enlistment cases and increase progressively for the cases included and 
the excessive-rate cases. The possible significance of this character- 
istic must await further analysis of the data before it can be evaluated. 

Created Variables 



Two summary advertising variables were created. One was the sum 
of all national media (excluding newpapers and supplements) used for 
the general enlisted program, and the other was a sum of the total 
national media, LAMS, and RAD expenditures. The first will be referred 
to as total media, and the latter, total advertising. In order to 
evaluate the continuing effects of advertising, the variables defining 
total advertising, total media, LAMS and RAD were lagged for one and 
two months. Exploratory analysis showed that there would be no signif- 
icant relationships between advertising and total enlistments beyond 
the two-month period. Lagging the advertising variables also meant 
that only 22 months could be studied for variables lagged one month, 
and only 21 months could be studied for those lagged for two months. 
Thus, reduction in sample size was also a tradeoff to be considered 
in lagging the advertising variables. Finally, the effects of lags 
cannot be evaluated fully for the LAMS and RAD variables, since a 
single original value for a quarter was spread over three months. 
Therefore, these variables lagged one month were the same as two 
current-month values; lagged two months, they were the same as one 
current-month value. This situation would tend to underestimate the 
lagged effect of LAMS and RAD. 

A dummy variable was created to account for the apparent dif- 
ferences in 1976 and 1977 in the distribution and total amounts of 
advertising expenditures. The dummy took a value of 1 for 1976 and 
a 0 for 1977. 
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**Cases with enlistments /QMA equal or greater than .005. 



The QMA for the county was multiplied by the unemployment rate 
to create the number of unemployed QMA in the county. 

A canvasser’s per goal variable was also created to account for 
possible differences in the work effort required to produce the goal. 

The original and created variables are shown in Table 10 with 
their means and standard deviations. 

Normalization and Transformation of Variables 



The simultaneity problem is not unique to the civilian sector. 

In Navy recruiting, a primary management function is to allocate 
goals and resources in a manner that ensures that the effort necessary 
to produce a recruit is equitable throughout the organization (Arima, 
1978). The basis for accomplishing this is the target population 
(basically, the QMA) in a command or unit. Accordingly, resources, 
goals, production, and the target population should all be highly 
inter correlated. The resources could include advertising. Table 11 
shows the correlations among enlistments, advertising, goals, can- 
vassers, QMA, high school graduates, and unemployed QMA. Obviously, 
the assertion is confirmed by the generally high intercorrelations 
evident in Table 11. A modification was necessary to eliminate the 
underlying mediator of the correlations. This was accomplished by 
using the QMA as a normalizer for the enlistment and advertising vari- 
ables, including all of the lagged variables. The correlations among 
the normalized variables is shown in Table 12. The actual unemploy- 
ment rate (UNEMP) was used in this table. 

Unfortunately, dividing by the QMA exaggerated the nonnormality 
in the variables and differences in the shapes of the distributions 
among the variables to such an extent that it was obvious that a linear 
model could not be fitted to the normalized data. A transformation 
that took the fourth root of the advertising variables before normaliz- 
ing by the QMA corrected this problem. These will hereafter be 
referred to as transformed variables and will be identified by a 
"4” inserted in their code names. The code names for data processing 
and the overall means and standard deviation of these normalized and 
transformed variables are given in Table 13. A comparison of the 
ratio of standard deviations to means in Table 13 with the raw measures 
in Table 10 will show the great improvement in the data for the fitting 
of a linear, least squares model. 

The correlations among the key recruiting, advertising, and 
situational variables after transformation and normalization are shown 
in Table 14. There it can be seen that the pattern of correlations 
is considerably different from the pattern in Table 11. There is not 
the evidence of management controls when the variables are normalized 
by the QMA, which is the primary tool of management. The uniquely 
high correlation in the table is total advertising with total enlist- 
ments. Examination of the other advertising variables showed 
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Y OF VARIABLES IN THE ENLISTMENT MODELS 
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Number Unemployed QMA UNEMQM 1452.8 3590.5 



CORRELATION AMONG VARIABLES REPRESENTING RAW DATA 
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Table 13 



SUMMARY OF VARIABLES AFTER NORMALIZATION AND TRANSFORMATION 



VARIABLE DESCRIPTION 


REGRESSION NAME 


MEAN 


STANDARD DEVIATION 


Enlistments /QMA 


ENLQMA 


.0011 


.0009 


Graduate Enlistments above 
50th %/QMA 


GR50QM 


.0005 


.0006 


/Total Media/QMA 


MED4QM 


.0013 


.0012 


^ t . 

/Total LAMS/QMA 


LAM4QM 


.0010 


.0009 


4 . . 

/Total RAD/QMA 


RAD4QM 


.0016 


.0014 


. 

/Total Advertising/QMA 


AD4QM 


.0019 


.0017 


Q GOAL/QMA 


GOLQMA 


.0005 


.0002 


Number Canvassers /QMA 


CANQMA 


.0003 


.0000 


Number Graduates/QMA 


GRADQM 


.2586 


.0877 


Unemployment Rate 


UNEMP 


.1823 


.0093 
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Table 14 
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1.00 



relationships similar to that for AD4QM in Table 14. This was to be 
expected because they are a component of total advertising. The 
correlations ranged from .69 to .73. The implications of these cor- 
relations and their impact on the fitting of a linear model will be 
developed in the results section. 

Model Fitting 

The stepwise, multiple-regression program of the Statistical 
Package for the Social Sciences (SPSS) (Nie, et al. , 1975) was used 
to fit the data. The regression was allowed to continue as long as 
the F statistic for the next variable to enter was significant at the 
.01 level and as long as the adjusted R square continued to increase 
with the entry of the variable. The regression with total (ENLQMA) 
and quality (GR50QM) enlistments was performed with total (ENLQMA) 
and quality (GRSOqM) enlistments as the dependent variables. The 
lagged advertising variables are identified with the numbers 1 or 2 
in the variable name to represent one or two months of lag. 

The model being fitted was the following: 



ENLQMA 

GR50QM 



a + + 1 * 2 X 2 + 




( 1 ) 



where 



= AD4QM 
X 2 = MED4QM 
X^ = LAM4QM 
X^ = RAD4QM 
X^ = AD14QM 
Xg = MED14QM 
X^ = LAM14QM 
Xg = RAD14QM 



a = constant 



Xg = AD24QM 

X^Q= MED24QM 

X^^= LAM24QM 

X^2= RAD24QM 

X^2= GOLQMA 

X^^= CANQMA 

X^^= GRADQMA 

X,,= GOLCAN 
Id 

x^2= UNEMP 



e , 



residual 



and the b are coefficients to be fitted from the data 



49 



RESULTS 



Exploratory Efforts 



The effort to fit the model given in equation (1) was unsatis- 
factory. While the overall fit was quite good (adjusted R^ = .556), 
the coefficients of the variables remaining in the equation were im- 
plausible and difficult to interpret. For example, total advertising 
lagged two months was the first to enter the equation and maintained 
a strong influence among the variables. Considerably later, however, 
current total advertising entered the equation with a negative sign. 

It was obvious that the multicollinear ity caused by the inclusion 
of the total advertising variable in the equation along with its 
component variables made the fit unstable and fortuitous. An examin- 
ation of Table 14 will show that the correlation of AD4QM with its 
component variables — MED4QM, LAM4QM, and RAD4QM — is .96, .94, and .98, 
respectively. The choice was to fit the equation with only the total 
advertising variables or without them. The latter alternative was 
chosen because it would be more informative to examine the effects of 
the separate advertising categories unconstrained by the domination 
of the total advertising variable. In addition, it had already been 
shown that the relationship of total advertising to enlistments was 
considerable as evidenced by a first-order correlation of .73 (Table 
14). Accordingly, the model was reexamined without the three vari- 
ables generated by total advertising, and the results are presented 
below. 

Total Enlistments (ENLQMA) and Advertising 



The results of the stepwise multiple regression of ENLQMA on 
the candidate independent variables are shown in Table 15. The 
adjusted R^ is .556 with an ^ ratio of over 3419 for 11/30,071 
It is highly significant statistically. The standard error of estimate 
is very small. The 11 variables in the equation are able to capture 
56 percent of the variance in 30,083 county-month cases in which new, 
male accessions to the Navy had occurred. Considering the large 
number of variables that have been suggested as influencing the 
enlistment of new recruits in the Navy, the power of this equation 
is certainly respectable. 

The variables in Table 15 are listed in their order of entry. 

The ^-at-entry is an evaluation of the incremental value of the vari- 
able when it entered the equation. The terminal ^ is the incremental 
value calculated for each variable when all other significant variables 
are already in the equation. Thus, the size of the terminal provides 
an indication of the unique contribution of a variable. The column of 
coefficients is for the equation in data form, while Beta is the 
coefficient when all variables are standardized — i.e., have a zero 
mean and unit standard deviation. The raw coefficients are not com- 
parable without a consideration of their scaling factors. On the 
other hand, the Beta coefficients are directly comparable. 
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Table 15 



STEPWISE MULTIPLE REGRESSION OF TOTAL ENLISTMENTS (ENLQMA) 



VARIABLE 


COEFFICIENT 


BETA 


F AT ENTRY 


TERMINAL F 


LAM4QM 


,20 


.22 


32,237 


134 


RAD4QM 


.08 


.14 


1,378 


71 


GRADQM 


.0009 


.09 


587 


473 


ME24QM 


.05 


.07 


242 


42 


RD24QM 


.12 


.20 


66 


141 


YR 


,00009 


.05 


77 


108 


CANQMA 


,51 


.03 


57 


55 


MED4QM 


.05 


.07 


42 


44 


LM24QM 


.14 


.12 


38 


55 


UNEMP 


-.00002 


-.02 


23 


24 


LM14QM 


-.11 


-.12 


22 


22 


Constant 


.0004 










Adjusted R Square 


“ .556 


F - 3419.4 






Standard Error 


= .00057 


d.f. = 11/30071 
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The notable feature of the model fitting is the early appearance 
of each of the advertising categories in the equation. Current LAMS 
(LAMAQM) enters first and continues to have the most influence on 
total enlistments. Total media enters first when it is lagged for 
two months (MED2AQM) . Of the lagged variables, only those lagged 
two months have a positive effect on enlistments. The only variable 
entering the equation when it is lagged one month is LAMS, and it 
enters with a negative weight. The only other variable with a nega- 
tive weight is unemployment. The other variables in the equation are 
GRADQM, YR, and CANQM, among which the high-school graduates variable 
provides the greatest effect. The year variable has a relative unique 
contribution to the equation. The variables that are absent, in 
addition to the advertising variables lagged one month, are those 
relating to goals: CANGOL and GOLQMA, 

Quality Enlistments (GR50QM) and Advertising 

The results of the stepwise multiple regression of quality 
enlistments (GR50QM) are shown in Table 16. Here, the adjusted R^ 
is ,737 with an ratio of 5,270 which, for 10/20, 6A5 , is 

statistically highly significant. In this instance, 10 variables 
enter the equation and are able to account for 7A percent of the 
variance in 20,656 county-month cases in which at least one indi- 
vidual who was a high school graduate with an AFQT score in the upper 
half of the distribution was enlisted in the Navy. Again, considering 
the many factors that have been named as affecting enlistments, it is 
noteworthy that, in the case of these priority, quality enlistees, an 
even greater proportion of the total variation in county-month acces- 
sions can be accounted for. 

The variables that enter the equation, their order of entry, 
and their relative effects are similar to those found for total enlist- 
ments (Table 15). The LAMS variable, lagged one month, which entered 
the previous equation with a negative coefficient does not appear in 
this equation, and canvassers per goal (CANGOL) replaces the canvassers 
per QMA variable. Again, goals per QMA is not a significant factor. 

Summary Results 

The results of this model fitting can be expressed in the two 
equations, below, that predict enlistments per QMA (equation (2)) and 
the quality enlistments per QMA (equation (3)). 

Total Enlistment Rate 

ENLQMA =,000A +,0009GRADQM .00009 YR + .5 CANQMA 
+ .20 LAMAQM + .08 RADAQM + .05 MEDAQM 
+ ,1A LM2AQM + .11 RD2AQM + .05 ME2AQM (2) 

Enlistment Rate of Quality Enlistees 

GR50QM = .0001 + , OOOA GRADQM + ,00008 YR 

+ .16 LAMAQM + .08 RADAQM + .06 MEDAQM 
+ .10 LM2AQM -f ,12 TF2AQM + ,0A ME2AQM (3) 
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Table 16 



STEPWISE MULTIPLE REGRESSION OF QUALITY ENLISTMENTS (GR50QM) 



VARIABLE 


COEFFICIENT 


BETA 


F AT ENTRY 


TERMINAL F 


LAM4QM 


.16 


.20 


A7,507 


173 


RADAQM 


.08 


.lA 


1,977 


81 


ME2AQM 


.OA 


.06 


AlO 


33 


GRADQM 


.OOOA 


.05 


166 


161 


YR 


.00008 


.06 


1A6 


173 


RD2AQM 


.12 


.23 


lAO 


200 


MEDAQM 


.06 


.10 


88 


88 


LM2AQM 


.10 


.13 


73 


76 


UNEMP 


-.00001 


-.01 


12 


8 


CANGOL 


.00003 


.01 


8 


8 


Constant 


.0001 

Adjusted R Square 
Standard Error 


- .737 
= .00035 


F - 
d.f. - 


578A.8 

10/206A5 
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DISCUSSION 



Study Methodology 

The meticulous and tedious effort required to acquire, sanitize, 
and coordinate pieces of data from many different sources so that they 
provided a comprehensive picture of the many simultaneously operating 
factors thought to affect recruiting at a very low level of aggregation — 
a county for one month — has apparently provided a very reliable and 
revealing assessment of the relationship between Navy advertising 
variables and recruiting production. It is necessary, however, to 
examine the methods used in more detail to evaluate the possible 
fortuitious occurrences and biases that might have played a part in 
the findings and to bring out ways in which the methods could be im- 
proved in future study efforts. 

First, the limitations of the data and the assumptions and 
procedures that were used in order to assign values for many variables 
to a specific county will be examined with a particular interest, at 
this point, in how they might have influenced the findings. The only 
data used in the study that actually originated at the county level 
were enlistments and the number of high-school graduates. In addition, 
a portion of the magazine circulation data was based on audited in- 
county circulation figures. The television and radio expenditures 
were also distributed on the basis of in-county survey statistics of 
audience viewing/listening habits. Almost half of magazine dollars 
were, however, distributed to counties from state circulation data on 
the basis of Sales Management 1977 population statistics. The outdoor 
advertising and the LAMS and RAD dollars were distributed to counties 
on the basis of the QMA. Moreover, the QMA is, itself, assigned to 
counties in an even more arbitrary manner from 400 "large” counties 
that were used to update the 1970, census-based QMA. That is, the 
country was divided into 400 areas that were studied for population 
trends and the trends were then used to update the QMA for each of 
the counties within one of the larger areas. Considerable solace can 
be taken from the fact that the QMA used in this study and high-school 
graduates in 1976 had a correlation of .97 (Table 11). Accordingly, 
the current QMA does seem to be a reasonable measure of the manpower 
pool for potential enlistees in a county, and its use as a normalizing 
variable in this study is sustained. The results obtained in this study 
are, however, valid only to the degree that the variables distributed 
on a population basis are in fact distributed that way in the real 
world. If, for example, LAMS dollars are used within an NRD to pur- 
chase listings of the recruiting stations in all telephone book 
yellow pages then it would be reasonable to assume that the monies 
were distributed on a population basis. On the other hand, if LAMS 
dollars were used for advertisements in school newspapers at certain 
large metropolitan high schools within an NRD, then such dollars 
might not be distributed evenly over the district’s counties. The 
ultimate solution to this problem is to set up data systems so that 
they can readily yield actual placement of advertising dollars at a 
very local level. 
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Another possible drawback in this study was the need to distribute 
LAMS and RAD expenditures temporally — i.e., spread quarterly values 
over three months equally. Not only did this necessity make it dif- 
ficult or impossible to obtain a valid assessment of these variables 
lagged in time, but the practice tended to increase multicollinearity 
in the data. For example, the current, one-month lagged and two- 
month lagged LAMS and RAD variables were identical in every third 
month of a quarter. This multicollinearity in the data seems to be 
the explanation for the negative coefficient for LAM14QM in the total 
enlistments equation (Table 15) and for total advertising lagged one 
month (AD14QM) in the exploratory analysis (Nie, , 1975). In 

addition, the high inter correlation among the three advertising cate- 
gories — media, LAMS, and RAD — also contributed to multicollinearity 
(Table 11) . A partial solution to the problem would be to collect 
data routinely at a finer temporal interval. For example, the monthly 
enlistment summary prepared by the recruiting command has enormous 
detail with respect to the recruiting effort, but it includes no 
advertising or promotions data. Were advertising data reported along 
with the other data, it would facilitate greatly the routine monitoring 
and tracking of the relationship between advertising and production. 

In passing, however, it should be noted that the significant part 
played by the variables lagged two months must be even greater in 
reality, since the method required in this study probably produced an 
underestimate of lagged effects. 

An effort was not made to differentiate the effects of the 
national media because, as shown in Tables 4 and 5, there was such 
great variability among months in the two-year period in the utiliza- 
tion of the media. Only magazines, followed by direct-mail advertis- 
ing, are reasonably represented throughout the study period. But 
seasonality in advertising expenditures and the advent of paid radio 
and television advertising at different points in the study period 
virtually made it impossible to enter the media independently in the 
model being fitted. The year dummy, however, is a broad indicator of 
advertising without (1976) and with (1977) paid, electronic media. It 
played a small, but unique part in the fitted equations for both 
total enlistments (equation (2)) and quality enlistments (equation 
(3)). Table 17 shows the media expenditures, average monthly canvasser 
strength, and the output of total and quality enlistments for 1976 
and 1977. The input variables, canvassers and media expenditures were 
considerably higher for 1977, but the output of the system in new 
recruits was considerably smaller — almost by 20 percent for the quality 
recruits. A better assessment of the differential relationships 
between national media expenditures and recruit production could 
be made by using only those months in which all media are represented. 
The scope of this effort did not permit analysis in that degree of 
detail. 

Another problem encountered in the study was the matter of time 
coincidence in the LAMS and RAD expenditures with the actual advertise- 
ment or promotional item appearing in the field. At the NRD level, it 
is possible to determine when a specific item of advertisement was 
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Table 17 

Comparison of Media Advertising with and without Paid Electronic Media 
and Concurrent Goal Accomplishment 
Item 1976* 1977 

Total Q enlistments 101,568 88,556 

Quality enlistments 51,309 37,495 

Canvassers** 3,212 3,360 

Media expenditures 3,916,371 5,803,669 

*Includes one month of paid radio advertising; includes 
December 1976. 

**Average per month strength. 
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scheduled to appear through pruchase-order files. If the commitment 
of funds is taken as the effective data, it will predate the actual 
advertisement; if the payment is the effective date, the actual appear- 
ance of the advertisement will be postdated. If the funds are just 
programmed monies, then it is not possible to tell when advertisements 
actually appeared. The RAD items, in addition to similar problems, have 
the additional problem that they are ordered merely to maintain in- 
ventory; the actual date of use is not determinable by any records. 

Thus, spreading quarterly data equally over three months probably 
tended to average out (smooth) any idiosyncracies in the data. On 
the other hand, when they were used as lagged variables, there was 
undoubtedly considerable deviation of the actual advertising from the 
precise month in which they were assumed to have appeared. 

The foregoing paragraphs concerned problems in the data, primarily. 
There were also arbitrary, study decisions that limit the generality of 
the results. First, there was the item of zero enlistments and exces- 
sive enlistment rates in counties that were excluded from the model 
fitting. VJhile it was shown that the smallness of the counties was a 
factor in their being outliers, there was also a suggestion that 
differential rates of advertising — at least for the excessive-rate 
outliers — might have been related to their being outliers. The 
results of the study, as conducted, would tend to reinforce this 
possibility. A different model could be used to examine the relation- 
ship further. Finally, being exploratory, this was a single-equation 
study and causal inferences cannot be made unambiguously. This does 
not mean that a tight, two-stage analysis for causal attribution, 
as described by James and Singh (1978) is possible, but an attempt 
should be made. The data are sufficient, however, to make a simul- 
taneous equation, recursive analysis as described by Aaker and Kay (1974) 
to permit better inferences about the directionality of the effects. 

Finally, it is obvious that the methods used cannot provide any 
indication of the appropriateness and effectiveness of the message 
being delivered. As the author was once told, ”What you say makes a 
difference, too.” But, as explained in Part I of this study, the 
message and the appropriateness of its reach and frequency belong to 
a set of participants in the advertising and marketing world that is 
quite apart from those who worry about expenditues and the bottom 
line. Nonetheless, the two areas must be joined to make valid infer- 
ences about the effectiveness of advertising. 

Advertising Effectiveness and Its Evaluation 

As stated, this study cannot assess the directional (causal) 
relationships between advertising and recruit production. But it can 
make inferences from the relationships seen between them. Specifically, 
the changes that take place in the relationships between advertising 
and production in the gross (raw) variables, normalized variables, and 
transformed variables will be examined as a process that could reveal 
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the adequacies and inadequacies in the allocation of advertising 
resources. It should be emphasized, at this point, that the fit of 
the linear model is only adequate when the actual allocation of adver- 
tising has been changed drastically by the transformation used. 

To begin with, gross (raw) advertising dollars — along with goals, 
canvassers, and enlistments — are closely related to the QHA, as 
shown in Table 11. The average correlation (using Fisher *s transfor- 
mation) is .72 between the four advertising categories in the table and 
the QMA, It should come as no surprise that populous counties are 
provided more advertising dollars than small counties. Slightly more 
than 50 percent of the variability in advertising dollars per county- 
month can be predicted on the basis of the Q^IA, alone. This leaves 
approximately half of the variability unexplained. Table 11 also 
shows that total enlistments are even more highly related to the QMA 
(r = .92), This implies that, while both gross advertising expendi- 
tures and gross enlistments are related to the QMA, they are not 
related to it in the same degree or, perhaps, in the same manner. 

This seems to account for the considerably smaller average correlation 
of .65 between the advertising categories and enlistments in Table 11. 

When the advertising and enlistment variables were normalized 
for their common QMA, the intercorrelations between them became 
essentially zero (Table 12), thus confirming the above-made suggestion 
that they related in a qualitatively different manner to the QMA. 

Since enlistments and advertising were originally correlated to a 
considerable degree and both were highly correlated with the 
QMA, it was obviously necessary to transform one of the variables to 
recapture the original relationship. As described in the methods 
section, taking the fourth root of advertising before dividing by the 
QMA reestablished linearity in the variables so that, as shown in 
Table 13, the average correlation between enlistments per QMA and 
the advertising variables per QMA returned to .72. 

The question with respect to advertising is, ’’Why does the re- 
lationship between advertising and production break down completely 
when they are expressed as rate variables with a common index?” The 
answer, in general terms, would seem to be that enlistments per 
QMA are fairly constant compared to advertising per QMA, which is 
more variable and tends to be large for the QMA when enlistments 
are not. The specific question then becomes, ’’Under what circumstances 
does this discrepancy between enlistments and advertising — with respect 
to the QMA — occur?” 

The implications of the fourth-root transformation for advertis- 
ing can be seen in Table 18, where contingency tables between enlist- 
ments per QMA and total advertising per QMA are presented for the case 
before (Table 18a) and the case after (Table 18b) the transformation. 
Before transformation (Table 18a), advertising expenditures are in- 
discriminately high with many cases of high advertising rates being 
associated with low enlistment rates. This is clearly evident when the 
lower-right and upper-right quadrants of the bef ore-t ransf ormat ion 
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JOINT DISTRIBUTION OF TOTAL ENLISTMENTS /QMA WITH TOTAL ADVERTISING/QMA 
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Total Adv v/Total Adv 



matrix are compared. Expenditures in the lower-right quadrant are 
roughly five times higher. 

As seen in Table 18 (b) , the fourth-root transformation com- 
presses the range of the advertising variable and moves the many 
cases from the pretransformation, lower-right quadrant to the lower- 
left, where they belong. That is, large advertising rates are no 
longer going into counties with bad enlistment rates. At the same 
time, the formerly very large advertising rates which are still 
represented in the right half of Table 18 (b) are now associated with 
the high enlistment rates in the upper-right quadrant, where they 
belong. The transformation necessary to achieve this condition indi- 
cates that high advertising rates per QMA are indiscriminately high 
and tend to be made on poor-producing areas to a greater degree than 
on the better or exceptionally well producing areas. The fit appears 
to be better for low advertising rates, in that they are associated 
more closely with the low producing counties rather than the high. 

Also, a smaller number of very high advertising rates are appropriately 
related to high-producing areas. 

The reason that this condition exists may be that the Recruit- 
ment Development Index (RDI), which is a measure of the relative 
potential of an area for enlistments, may be quite large compared to 
the counties used as a basis for analysis in this study. Since the 
RDI is used as a basis for the allocation of resources, an area with 
a moderate or even high RDI may have significant pockets of very poor 
potential within them. Nevertheless, these poor areas share the 
largesse in advertising that appears to be warranted for the entire 
area. Further research with the data available in this study could 
identify the exact county-months where the enlistment rate is poor 
but the advertising rates are high. Since the LAMS and RAD advertis- 
ing categories — which are the easiest to control geographically — are 
also the most closely associated with the enlistment rates in the fitted 
equations, and especially so with the quality enlistees (Table 16), it 
would appear that an effort should be made to identify those counties 
where the large discrepancies occur. It could also be argued that 
these discrepant cases are artifacts caused by the necessity to allo- 
cate NRD-level data to counties in proportion to their QMA; in 
actuality, those NRD resources might be more discretely and effectively 
allocated. This possibility could be investigated by comparing the 
present data with the actual data that should be available at the NRD 
level. For example, it would not be difficult to identify where large 
LAMS expenditures are not accompanied by high enlistment rates and 
then go to the NRDs involved and check the purchase order files. 

A more theoretical question posed by the foregoing analysis is 
to what degree should advertising expenditures track the enlistment 
rate? Should they be greater in areas with poorer enlistment rates 
to develop the enlistment potential in the area, or should they be 
greater where the enlistment rate is high to ensure continuation of. 
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and even improve, those rates? The answer to this question would be 
difficult to obtain using naturally occurring data because the condi- 
tions necessary to answer the question would occur too infrequently. 

The alternative would be well controlled levels experiments. 

The nonadvertising variables that helped raise the multiple- 
regression estimates of enlistment rates over the first-order correla- 
tions of advertising with enlistments should be mentioned. Of these, 
the effects of the year dummy have already been examined in the dis- 
cussion of study methodology, above. Another variable with a signi- 
ficant effect was high school graduates per QMA. Since the correla- 
tion of the raw data for high school graduates and QMA is .97, the 
value of the variable should have been rather constant. (The ratio 
of the standard deviation to the mean as seen in Table 10 is 1:3.) 
Nevertheless, it enters both equations and makes a modest but signif- 
icant contribution to the predicted enlistment rates (Tables 15 and 
16). Thus, the quality of an area as defined by the ratio of high 
school graduates to the QMA contributes to higher enlistment rates. 

This is consistent with Arima^s (1978b) finding that the best, single 
predictor of individual recruiter productivity was a high school 
enrollment measure. Moreover, he found that a combination of educa- 
tional variables predicted 35 percent of the variability in individual 
recruiter productivity. 

The other variable to enter the equations was a measure of the 
variability in canvasser strength per county with respect to a 
county’s QMA or recruiting goals (Q goals). The intercorrelation 
among canvassers, goals, and the QMA ranges from .92 to .98 in Table 
11 and are obviously driven by the QMA. Thus, it would not seem to 
make a great deal of difference whether the variable is canvassers 
per QMA or per goal. In either case, the contribution to the equation 
is small, but statistically very significant. As small as the varia- 
tion in canvassers per QMA seems to be in actuality, it is important 
to note that, when a rate variable — such as enlistments per QMA — is 
used as the production variable, the latter is very sensitive to 
variability in the distribution of canvassers with respect to goals 
or the QMA. 

Finally, a variable that is very significant by its failure to 
enter the regression equations is goals per canvasser. It is generally 
assumed that goals drive the recruiting system and a rate variable, such 
as enlistments per QMA, ought to be very sensitive to another rate 
variable such as goals per QMA. While nonlinearity in these rate 
variables could have been a factor contributing to this lack of effect, 
there are other cogent reasons. With the extension of the delayed 
entry program (DEP) to one year, the importance of goals is mediated 
by the status of DEP enlistments for a particular NRD. If the DEP 
pool is well filled, goals will not have a significant effect on current- 
month canvassing activity. If the goal for the current month can be 
readily met but the DEP pool is severely depleted, current-month canvas- 
sing may very well be quite active. To state the case in another way. 
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it was formerly acdknowledged that goals could be a lid on production, 
but this is no longer the case when the DEP pool also requires filling. 
The goal may be a lid on current production when it is defined as 
shipments to the RTC, but not on production defined as numbers taking 
the oath of enlistment — i.e., contracts signed, as used in this study. 
Thus, it would appear that an uncontrolled variable that could have 
accounted for a significant portion of the unexplained variance — and 
would have to be controlled or used as a covariate for any levels 
experiment — is a relationship between goals and the DEP pool for a 
given organization (NRD) . This would be a difficult variable to 
quantify, since a time function is also involved. That is, the DEP 
status 12 months from the current month does not have the urgency 
that the DEP status of the next month has. Moreover, the urgency of 
the DEP status in the out months is also determined by the projected 
goals for those months. It may be that good and bad management of the 
recruiting effort could be very closely related to the way that the 
current effort is managed with respect to the goals/DEP relationship 
in the out months. Unfortunately, no measure exists to quantify the 
drive that this relationship should — or does — provide for the recruiting 
effort. 
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SUMMARY AND CONCLUSIONS 



This part of the study was an empirical examination of the 
relationship between advertising and enlistments while taking into 
account the allocation of recruiters, assignment of goals, the popula- 
tion of high school graduates and qualified military availables (QMA) , 
and the monthly, national, unemployment rate for young people. Values 
for these variables were obtained, assigned, or allocated to each 
county in the contiguous United States. The unit of analysis was 
by county by month for calendar years 1976 and 1977 of those county- 
months that had at least one enlistment. In addition, advertising 
variables were lagged for one and two months, and enlistments reflected 
quality as well as quantity of first-term male enlistees. 

The principal findings were: 

1. The allocation of advertising resources, canvassers, and 
goals and the production of enlistments were highly interrelated and 
all were closely related to the county’s QMA. These relationships 
were a reflection of managerial planning and not of independent 
cause-effect relationships. 

2. When the variables were normalized (indexed) by the QMA 
and made into rate variables that reflected quality as well as the 
gross quantity aspects of recruiting operations, the relationships 
seen in the gross (raw) data were highly attenuated and the linear 
relationship between advertising and enlistments disappeared. 

3. In general, increasing advertising rates were not accompanied 
by any systematic change in enlistment rates. In particular, higher 
advertising rates were accompanied by a greater frequency of counties 
with low, rather than high, enlistment rates. 

4. A fourth root transformation of the advertising variables 
before division by the QMA was necessary to establish linearity between 
enlistment rates and advertising rates to permit the fitting of a linear, 
least-squares regression model to the data. 

5. The fitted equations accounted for 56 percent of the varia- 
tion in total enlistment rates and 74 percent of the variation in the 
enlistment rates of quality enlistees. The transformed LAMS and RAD 
variables in their current and lagged two-months versions had the 
greatest effect; the transformed national media rates, especially 
lagged two months, also affected the enlistment rate. The dummy 
variable for year (1976, 1977) and the population of high school 
graduates per QMA provided moderate, but unique influences on the 
enlistment rate. 

6. The significant year dummy provided an opportunity to assess 
the effects of a major change in media mix — with and without paid, 
electronic media advertising. In 1977, with more advertising dollars 
and canvassers and the full implementation of paid radio and television 
advertising, enlistments were considerably lower. 
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7. Utilization of the Recruitment Development Index (RDI) by 
the recruiting command and its advertising agency for planning and 
resource allocation does not seem to be working well. The problem, if 
one exists, could be because the size of the area used in developing 
the RDI may be too large so that pockets of poor potential within a 
otherwise high-potential area receive relatively too much advertising. 
Another possibility is that the enlistment rate used in the RDI is 
based on NRD production in terms of shipments to the RTCs (instead 

of contracts signed). Shipments are primarily a function of manage- 
ment decision and, as shown in 1 and 2, above, do not reflect the 
true production of an area with respect to its potential. 

8. The goals variable was highly related to gross production 
but, when indexed by the QMA, it did not influence the enlistment 
rate, as expected. It was suggested that goals must be related to 
the DEP status of a unit before they play a significant role on the 
effort to obtain new contracts. Since the DEP pool extends for one 
year in time, a measure that combines goals and DEP status into a 
single variable that is a measure of the force that drives enlistments 
will be complex and difficult to create, but it is sorely needed for 
management and research programs. 
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ABBREVIATIONS 



ABC 


Audited Board of Circulation 


AFQT 


Armed Forces Qualification Test 


ARB 


Advertising Research Bureau 


ASVAB 


Armed Service Vocational Aptitude Battery 


AVF 


All-volunteer Force 


CNRC 


Commander, Navy Recruiting Command 


DEP 


Delayed Enlistment Program 


DMDC 


Defense Data Management Center 


HumRRO 


Human Resources Research Organization 


LAMS 


Local Advertising Management System 


NALTS 


Navy Lead Tracking System 


NCFA 


Navy Campus for Achievement 


NNRIC 


National Navy Recruiting Information Center 


NOIC 


Navy Opportunities Information Center 


NRA 


Navy Recruiting Area 


NRD 


Navy Recruiting District 


QMA 


Qualified Military Available(s) 


QUEBEC 

("Q”) 


Male applicant at least 17 years of age without prior service 


RAD 


Recruiting Aids Division, also recruiting aids produced by 
Recruiting Aids Division 


RDI 


Recruitment Development Index 


RTC 


Recruit Training Center 


TAR 


Training and Administration of the Reserve 


USAREC 


U.S. Army Recruiting Command 


USN 


U.S. Navy (Regular Navy) 


USNR 


U.S. Naval Reserve 


USP 


Unique selling proposition 


VCS 


Visual Campaign Summary 
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